-- Logs begin at Mon 2022-08-01 11:21:46 UTC, end at Mon 2022-08-01 12:57:06 UTC. -- Aug 01 11:49:29.953748 dv-cinder-850320-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 01 11:49:29.963762 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 1 11:49:29 2022] *** Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: nodename: dv-cinder-850320-2-iscsi Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: machine: x86_64 Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: clock source: unix Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: pcre jit disabled Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: detected number of CPU cores: 4 Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: current working directory: / Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: detected binary path: /usr/bin/uwsgi-core Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: your processes number limit is 47680 Aug 01 11:49:29.966097 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: your memory page size is 4096 bytes Aug 01 11:49:29.966720 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: detected max file descriptor number: 2048 Aug 01 11:49:29.966720 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: lock engine: pthread robust mutexes Aug 01 11:49:29.966720 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: thunder lock: enabled Aug 01 11:49:29.966720 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 01 11:49:29.966720 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 01 11:49:30.005800 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: Python main interpreter initialized at 0x55b38ba4a9c0 Aug 01 11:49:30.005800 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: python threads support enabled Aug 01 11:49:30.005800 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: your server socket listen backlog is limited to 100 connections Aug 01 11:49:30.005800 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: your mercy for graceful operations on workers is 90 seconds Aug 01 11:49:30.006208 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: mapped 403077 bytes (393 KB) for 2 cores Aug 01 11:49:30.006208 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: *** Operational MODE: preforking *** Aug 01 11:49:30.006536 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: *** uWSGI is running in multiple interpreter mode *** Aug 01 11:49:30.006536 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: spawned uWSGI master process (pid: 80154) Aug 01 11:49:30.006764 dv-cinder-850320-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 01 11:49:30.008335 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: spawned uWSGI worker 1 (pid: 80163, cores: 1) Aug 01 11:49:30.008335 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: spawned uWSGI worker 2 (pid: 80164, cores: 1) Aug 01 11:49:30.008829 dv-cinder-850320-2-iscsi devstack@keystone.service[80154]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 01 11:49:35.486265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 11:49:35.486265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: from cryptography.utils import int_from_bytes Aug 01 11:49:35.487186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 11:49:35.487186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: from cryptography.utils import int_from_bytes Aug 01 11:49:35.487606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 11:49:35.487606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: from cryptography.utils import int_from_bytes Aug 01 11:49:35.487711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 01 11:49:35.487711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: from cryptography.utils import int_from_bytes Aug 01 11:49:35.536570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 11:49:35.536787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 01 11:49:35.536874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 11:49:35.536949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 01 11:49:35.536949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] command line args: [] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 11:49:35.537063 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] command line args: [] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 01 11:49:35.537158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 11:49:35.537256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 01 11:49:35.537326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ================================================================================ {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 11:49:35.537384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ================================================================================ {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 01 11:49:35.537627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] admin_token = **** {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.537723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] admin_token = **** {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.537802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] config_dir = [] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.537860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] config_dir = [] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.537976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.538058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.538172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] config_source = [] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.538321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] config_source = [] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.538321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] debug = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.538482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] debug = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.538554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.538817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.538962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] list_limit = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] list_limit = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] log_config_append = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.539984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] log_config_append = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.540130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.540214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.540281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] log_dir = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.540474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] log_dir = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.540541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] log_file = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.540610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] log_file = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.540675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.540733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.540893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.540977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.541113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.541204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.541347 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.541429 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.541574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.541666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.541752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.541831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.541903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.541976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.542157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.542242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.542373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.542456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.542676 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.542870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.542992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.543089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.543161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.543221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.543322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.543461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.543580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.543659 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.543823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.543926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.544329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.544414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.544526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] publish_errors = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.544718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] publish_errors = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.544801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.544868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.544938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.544999 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.545132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.545219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.545357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.545464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.545607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.545692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.545832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] standard_threads = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.545935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] standard_threads = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] use_journal = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] use_journal = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] use_json = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] use_json = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] use_stderr = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.546984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] use_stderr = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.547092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] use_syslog = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.547181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] use_syslog = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.547327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.547416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 01 11:49:35.547582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.547664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.547820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.547906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.548074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.548161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.548308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.548399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.548577 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.548658 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.548815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.548901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.549056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.549143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.549288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.549390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.549515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.549601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.549775 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.549857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.550024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.550107 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.550256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.550338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.550505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.550586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.550717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.550795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.550963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.551040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.551204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.551288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.551441 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.551515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.551678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.551765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.551929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.552012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.552173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.552257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.552432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.552509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.552688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.552819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.552944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.553024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.553194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.553272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.553436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.553514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.553666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.553752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.553895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.553979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.554123 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.554206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.554354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.554433 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.554598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.554675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.554843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.554929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.555073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.555149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.555299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.555375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.555544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.555632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.555785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.555863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.556022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.556102 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.556263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.556414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.556518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.556596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.556751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.556828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.556986 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.557148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.557227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.557296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.557427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.557504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.557659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.557745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.557886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.557962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.558132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.558313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.558415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.558502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.558647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.558730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.558879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.558946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.559116 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.559216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.559344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.559410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.559574 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.559678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.559820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.559952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.560074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.560171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.560266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.560415 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.560551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.560663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.560761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.560864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.560988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.561067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.561179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.561304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.561410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.561485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.561613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.561730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.561832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.561918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.562100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.562221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.562335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.562578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.562666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.562826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.563053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.563173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.563287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.563485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.563574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.563785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.563924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.564060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.564268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.564368 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.564783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.564864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.565021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.565261 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.565369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.565509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.565645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.565733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.565808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.565974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.566054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.566219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.566298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.566458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.566545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.566697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.566779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.567031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.567108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.567319 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.567486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.567565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.567819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.567937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.568055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.568338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.568463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.568563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.568809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.568887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.569050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.569296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.569387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.569533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.569632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.569769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.569855 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.570044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.570131 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.570231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.570471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.570579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.570720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.570937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.571029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.571214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.571446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.571580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.571889 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.571958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.572195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.572378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.572557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.572825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.572906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.573072 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] auth.external = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.573255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.573348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.573717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.573803 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.573986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.574106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.574223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.574360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] auth.password = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.574443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.574598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] auth.token = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.574686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.574846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.574934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.575091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.575171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.575347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.575425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.575585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.575695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.575821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.576013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.576100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.576373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.576463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.576658 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.576815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.576934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] credential.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.577248 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] auth.external = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.577328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.577528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.577639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.577791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.577890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.578031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.578111 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.578283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.578364 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] auth.password = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.578523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.578610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] auth.token = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.578768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.578847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.579008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.579093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.579256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.579353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.579489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.579569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.579738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.579818 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.579988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.580082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.580219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.580329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.580479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] credential.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.580560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.580727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.580820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.581045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] federation.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.581166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.581286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.581543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.581622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.581796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.582033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.582112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.582285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.582453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.582532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.582775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.582855 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.583015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.583251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.583332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.583504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.583663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.583771 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.584010 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.584089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.584263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.584404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.584544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.584676 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.584787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.584929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.585029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.585282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.585484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.585565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.585729 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.585809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.585988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.586066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.586221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] federation.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.586300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.586457 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.586537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.586694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.586819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.586978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.587045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.587166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.587246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.587411 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.587508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.587653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.587734 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.587910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.587989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.588144 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.588232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.588424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.588506 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.588664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.588756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.588905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.589040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.589140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.589219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.589399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.589478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.589690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.589782 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.589931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.590011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.590184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.590264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.590431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.590519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.590664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.590750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.590902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.590981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.591151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.591237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.591398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.591476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.591629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.591708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.591860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.591948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.592091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.592170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.592365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.592450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.592597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.592690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.592834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.592914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.593077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.593157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.593318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.593407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.593567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.593647 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.593807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.593886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.594040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.594127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.594271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.594350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.594507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.594593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.594737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.594826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.594967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.595045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.595206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.595284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.595444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.595533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.595681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.595759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.595908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.595987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.596144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.596231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.596413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.596493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.596649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.596729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.596900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.596991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.597133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.597212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.597361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.597440 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.597598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.597686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.597828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.597909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.598066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.598146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.598296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.598382 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.598535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.598615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.598770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.598876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.599002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.599083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.599236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.599316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.599481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.599560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.599713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.599794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.599944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.600024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.600193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.600378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.600476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.600635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.600715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.600874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.600953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.601125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.601213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.601352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.601430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.601590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.601669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.601827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.601914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.602068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.602149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.602311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.602437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.602558 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.602647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.602795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.602874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.603025 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.603104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.603254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.603341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.603487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.603567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.603721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.603801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.603956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.604043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.604183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.604261 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.604451 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.604532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.604697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.604789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.604934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.605014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.605201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.605350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.605431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] resource.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.605599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.605679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.605832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.605911 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.606081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.606167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.606318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.606400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.606554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.606634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.606794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.606874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.607029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.607110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.607274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.607361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.607524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.607604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.607758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] role.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.607838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.608009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] role.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.608088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.608233 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.608332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.608501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.608582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.608736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.608816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.608970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.609049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.609205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.609286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.609448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.609528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.609688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.609767 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.609922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.609999 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.610386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.610463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.610535 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] resource.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.610608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.610690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.610766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.610839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.610912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.611051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.611132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.611301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.611382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.611549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.611776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.611856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.612017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.612098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.612246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.612379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.612505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.612584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.612728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] role.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.612810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.612974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] role.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.613057 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.613204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.613283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.613452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.613532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.613698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.613778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.613936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.614015 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.614173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.614277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.614408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.614489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.614644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.614726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.615230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.615317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.615479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.615589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.615712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.615816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.615956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.616059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.616196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.616373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.616479 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.616575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.616689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] token.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.616790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.616895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.617124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.617234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.617359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.617478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.617607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.617716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.617853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.617960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.618094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.618213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.618318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.618419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.618563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.618662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.618807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.618914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.619044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.619148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.619277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.619380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.619488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.619588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.619723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.619828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.619954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.620066 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.620185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.620317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.620466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.620579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.620793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.620913 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.621039 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.621148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.connection = **** {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.621275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.621430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.621519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.621593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.621726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] token.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.621806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.621962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.622044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.622207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.622285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.622437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.622514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.622672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.622751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.622910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.622996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.623142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.623220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.623381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.623460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.623617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.623695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.623850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.623928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.624084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.624164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.624343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.624427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.624579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.624657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.624814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.624897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.625058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.625137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.625283 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.625361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.625443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80164) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 11:49:35.625516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.625803 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.626060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.connection = **** {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.626305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.626551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.626793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.627031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.627264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.627497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.627730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.627963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.628203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.628460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.628702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.628934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.629180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.629499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.629663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.629893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.630133 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.630376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 01 11:49:35.630545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80163) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 01 11:49:35.659114 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.659340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.659566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.660431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.660671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.660900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.668230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.669584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.698188 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.698448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.699557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.699791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.751555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.751738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.802140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.802349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.804234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.804480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.868241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.868493 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.868715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.868931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.869117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.869213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.869364 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.869460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.869604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.869795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.870037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.870240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.892477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.892782 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.892940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.893063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.893179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.893305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.893451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.893590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.893707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.893827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.893951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.894069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.894187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.894187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.904901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.905366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.917736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.918133 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.918469 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.918593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.918687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.918815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.918899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.919102 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.919226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.919296 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80163) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.919358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.919523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.919723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.919957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80164) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 01 11:49:35.936889 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.937044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.937044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 11:49:35.937246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 11:49:35.937404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.937538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.937538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 11:49:35.937782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 01 11:49:35.937919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.938012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.938087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.938182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.938301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.938369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.938545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.938686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.948122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.948252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.948394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.948488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.948688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.948948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.949038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.949103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.955948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:35.956938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:35.960858 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:35.961125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.961429 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.961683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.961989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.962249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.962515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.962754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:35.962883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.963041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.963256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.963383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.963568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.963693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.964222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.964590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.964838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.965266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.965756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.966237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.966735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.967222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.986662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.988173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:35.988476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.990917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.992429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:35.992713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.993486 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:35.993768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.994051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.997349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:35.997617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:35.997895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.000786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.001067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.004549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.004779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.006207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.006496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.006792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.007072 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.007345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.007637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.007900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.008183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.009049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.009300 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.009626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.009863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.010136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.010419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.010690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.010960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.024094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.024397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.027013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.027297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.028417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.031220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.031543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.031851 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.034348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.034631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.036218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.038981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.039465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.042144 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.042702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.043007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.043298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.043575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.043863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.044156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.045340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.045633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.045909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.046199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.046483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.046766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.060168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.060447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.060742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.061005 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.062712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.062996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.063264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.063517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.067901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.069470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.069722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.070384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.071901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.072176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.072592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.072875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.073175 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.073451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.073726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.075069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.075340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.075627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.075900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.076181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.085195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.085474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.085766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.086040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.086315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.086584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.086855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.087538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.087816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.088091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.088375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.088665 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.088927 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.089210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.102560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.104763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.105622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.107828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.108612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.110730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.111301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.112741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.113051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.113333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.113423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.115391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.115675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.115946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.120181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.121879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.122509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.124096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.124842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.125174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.125456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.125730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.127006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.127292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.127564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.127839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.133778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.134073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.134362 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.135822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.136107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.136439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.141245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80164) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.141519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.141800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.142075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.142355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.142632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.142903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.143163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.143269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80163) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 01 11:49:36.143457 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.143542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.143730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.143814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.144010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.144095 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.144307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.144408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.144591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.144676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.144850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80164) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.144937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.145108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.145470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.145749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.146031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.146280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.146551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.146809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80163) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 01 11:49:36.178197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b38ba4a9c0 pid: 80164 (default app) Aug 01 11:49:36.180463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b38ba4a9c0 pid: 80163 (default app) Aug 01 11:49:36.181665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8bc7d-1f4c-4684-842d-be19241c51e0 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:36.181956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8bc7d-1f4c-4684-842d-be19241c51e0 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:36.182227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8bc7d-1f4c-4684-842d-be19241c51e0 None None] PATH_INFO: `/v3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:36.186715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Mon Aug 1 11:49:36 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:43.857060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415ca22-4077-4f11-9e10-2ef628c3742b None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:43.857557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415ca22-4077-4f11-9e10-2ef628c3742b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:43.857557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415ca22-4077-4f11-9e10-2ef628c3742b None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:43.862128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:43.867901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:43.868397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:43.868490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:43.946398 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: INFO dbcounter [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] Registered counter for database keystone Aug 01 11:49:43.966429 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG oslo_db.sqlalchemy.engines [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80164) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 11:49:43.973850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writer thread running {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 11:49:44.153245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG passlib.handlers.bcrypt [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80164) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 11:49:44.153586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG passlib.handlers.bcrypt [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80164) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 11:49:44.209078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:44.210762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:44.355218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 11:49:44.501786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415ca22-4077-4f11-9e10-2ef628c3742b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:44.571557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: INFO dbcounter [None req-5415ca22-4077-4f11-9e10-2ef628c3742b None None] Registered counter for database keystone Aug 01 11:49:44.593406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG oslo_db.sqlalchemy.engines [None req-5415ca22-4077-4f11-9e10-2ef628c3742b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80163) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 01 11:49:44.600781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writer thread running {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 01 11:49:44.718438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415ca22-4077-4f11-9e10-2ef628c3742b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:44.723095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ce235-71f3-4a2a-9e0f-db5c86a02b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:44.723776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ce235-71f3-4a2a-9e0f-db5c86a02b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:44.724051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ce235-71f3-4a2a-9e0f-db5c86a02b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:44.724330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ce235-71f3-4a2a-9e0f-db5c86a02b24 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:44.733010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ce235-71f3-4a2a-9e0f-db5c86a02b24 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:44.784481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dc3ce235-71f3-4a2a-9e0f-db5c86a02b24 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 11:49:44.784713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ce235-71f3-4a2a-9e0f-db5c86a02b24 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:44.795143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Mon Aug 1 11:49:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 11:49:44.801445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:44.816642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4faf7-69f9-467b-b712-9c723ba1bc87 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:44.822574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5f593-0eeb-4106-a799-bdfa9fbd1599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:44.823863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5f593-0eeb-4106-a799-bdfa9fbd1599 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:44.824150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5f593-0eeb-4106-a799-bdfa9fbd1599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:44.824455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5f593-0eeb-4106-a799-bdfa9fbd1599 admin admin] PATH_INFO: `/v3/projects/a841fcca183a4d31ac0320c337760588` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:44.826094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5f593-0eeb-4106-a799-bdfa9fbd1599 admin admin] RBAC: Authorizing `identity:get_project(project_id=a841fcca183a4d31ac0320c337760588)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:44.876630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c3d5f593-0eeb-4106-a799-bdfa9fbd1599 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 01 11:49:44.877355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5f593-0eeb-4106-a799-bdfa9fbd1599 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:44.881977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Mon Aug 1 11:49:44 2022] GET /identity/v3/projects/a841fcca183a4d31ac0320c337760588 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:46.841768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351137a-e2e5-4039-a023-74f7cbe16609 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:46.842340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351137a-e2e5-4039-a023-74f7cbe16609 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:46.842340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351137a-e2e5-4039-a023-74f7cbe16609 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:46.846444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:46.852150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7674ee-be91-45ff-bb84-931586050100 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:46.852636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7674ee-be91-45ff-bb84-931586050100 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:46.852729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7674ee-be91-45ff-bb84-931586050100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:46.874867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8d7674ee-be91-45ff-bb84-931586050100 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:46.876449 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8d7674ee-be91-45ff-bb84-931586050100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:46.886226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 01 11:49:46.909272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351137a-e2e5-4039-a023-74f7cbe16609 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:46.925785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351137a-e2e5-4039-a023-74f7cbe16609 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:46.931451 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b94a8-4247-4c98-b47f-3a5f847446d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:46.932067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b94a8-4247-4c98-b47f-3a5f847446d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:46.932364 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b94a8-4247-4c98-b47f-3a5f847446d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:46.932636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b94a8-4247-4c98-b47f-3a5f847446d4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:46.941609 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b94a8-4247-4c98-b47f-3a5f847446d4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:46.943443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b94a8-4247-4c98-b47f-3a5f847446d4 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:46.952968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Mon Aug 1 11:49:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 11:49:46.960666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7674ee-be91-45ff-bb84-931586050100 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:46.971929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7674ee-be91-45ff-bb84-931586050100 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:46.977292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206b3f-d690-43b4-882f-f53bcfcd010f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:46.977938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206b3f-d690-43b4-882f-f53bcfcd010f admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:46.978221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206b3f-d690-43b4-882f-f53bcfcd010f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:46.978498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206b3f-d690-43b4-882f-f53bcfcd010f admin admin] PATH_INFO: `/v3/users/85a5af1109c54a2a80385f00893e3137` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:46.980039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17206b3f-d690-43b4-882f-f53bcfcd010f admin admin] RBAC: Authorizing `identity:get_user(user_id=85a5af1109c54a2a80385f00893e3137)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:46.981930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17206b3f-d690-43b4-882f-f53bcfcd010f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:46.983538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Mon Aug 1 11:49:46 2022] GET /identity/v3/users/85a5af1109c54a2a80385f00893e3137 => generated 247 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:49.042951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43370df-ffec-4fff-af7c-46d66f99a42b None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.043514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43370df-ffec-4fff-af7c-46d66f99a42b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.043514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43370df-ffec-4fff-af7c-46d66f99a42b None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.044878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:49.050316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7df02f-d786-4887-bbbd-9cca6cff5d53 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.050763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7df02f-d786-4887-bbbd-9cca6cff5d53 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.051019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7df02f-d786-4887-bbbd-9cca6cff5d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.054996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c994cbd-96ce-4a0d-8a11-76aad9796e95 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.055279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c994cbd-96ce-4a0d-8a11-76aad9796e95 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.055550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c994cbd-96ce-4a0d-8a11-76aad9796e95 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.056841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:49.062324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c33d3-4bfd-4205-963f-5497e4f059fb None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.062622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c33d3-4bfd-4205-963f-5497e4f059fb None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.062900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2c33d3-4bfd-4205-963f-5497e4f059fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.070847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-0e7df02f-d786-4887-bbbd-9cca6cff5d53 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:49.072056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0e7df02f-d786-4887-bbbd-9cca6cff5d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:49.106615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG passlib.handlers.bcrypt [None req-6c2c33d3-4bfd-4205-963f-5497e4f059fb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80163) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 01 11:49:49.106954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG passlib.handlers.bcrypt [None req-6c2c33d3-4bfd-4205-963f-5497e4f059fb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80163) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 01 11:49:49.153587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-6c2c33d3-4bfd-4205-963f-5497e4f059fb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:49.155230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6c2c33d3-4bfd-4205-963f-5497e4f059fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:49.158689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:49.181856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7df02f-d786-4887-bbbd-9cca6cff5d53 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:49.206808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:49.261479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:49.283586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c33d3-4bfd-4205-963f-5497e4f059fb None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:49.290145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7df02f-d786-4887-bbbd-9cca6cff5d53 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:49.329738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4cfda-3437-4b16-bcfa-b8a055a6497f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:49.330402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4cfda-3437-4b16-bcfa-b8a055a6497f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.330739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4cfda-3437-4b16-bcfa-b8a055a6497f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.331018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4cfda-3437-4b16-bcfa-b8a055a6497f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.343132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4cfda-3437-4b16-bcfa-b8a055a6497f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-88d4cfda-3437-4b16-bcfa-b8a055a6497f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:49:49.351981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:49:49.352976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:49:49.353762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:49:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:49:49.359736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4cfda-3437-4b16-bcfa-b8a055a6497f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:49.393213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2c33d3-4bfd-4205-963f-5497e4f059fb None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:49.395505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:49.431739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21daa456-fcd2-4a48-a113-9bff07eaecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:49.432701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daa456-fcd2-4a48-a113-9bff07eaecfc None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.432987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daa456-fcd2-4a48-a113-9bff07eaecfc None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.433265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21daa456-fcd2-4a48-a113-9bff07eaecfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.444611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:49.462991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4cfda-3437-4b16-bcfa-b8a055a6497f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:49.497331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b1e0f-fa2a-4984-a27f-307ec755da64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:49.497957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b1e0f-fa2a-4984-a27f-307ec755da64 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.498235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b1e0f-fa2a-4984-a27f-307ec755da64 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.498504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b1e0f-fa2a-4984-a27f-307ec755da64 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.499228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b1e0f-fa2a-4984-a27f-307ec755da64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:49:49.499523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b1e0f-fa2a-4984-a27f-307ec755da64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:49.500414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21daa456-fcd2-4a48-a113-9bff07eaecfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:49.500949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b1e0f-fa2a-4984-a27f-307ec755da64 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:49.501967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21daa456-fcd2-4a48-a113-9bff07eaecfc None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:49.512948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:49:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:49:49.518688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b1e0f-fa2a-4984-a27f-307ec755da64 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:49.605276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 8/16] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:49:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:49:49.611575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21daa456-fcd2-4a48-a113-9bff07eaecfc None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:49.615212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:49.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21daa456-fcd2-4a48-a113-9bff07eaecfc None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:49.738744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b1e0f-fa2a-4984-a27f-307ec755da64 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:49.761542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baf498-e2f4-4ce2-ab7a-854d4c1ae093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:49.762444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baf498-e2f4-4ce2-ab7a-854d4c1ae093 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.762727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baf498-e2f4-4ce2-ab7a-854d4c1ae093 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.763006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84baf498-e2f4-4ce2-ab7a-854d4c1ae093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.776767 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af052d-3764-4aa0-a0f2-9b179d99122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:49.777449 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af052d-3764-4aa0-a0f2-9b179d99122e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.777734 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af052d-3764-4aa0-a0f2-9b179d99122e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.777979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af052d-3764-4aa0-a0f2-9b179d99122e None admin] PATH_INFO: `/v3/users/85a5af1109c54a2a80385f00893e3137` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.779179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af052d-3764-4aa0-a0f2-9b179d99122e None admin] RBAC: Authorizing `identity:get_user(user_id=85a5af1109c54a2a80385f00893e3137)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:49.780709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af052d-3764-4aa0-a0f2-9b179d99122e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:49.781953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baf498-e2f4-4ce2-ab7a-854d4c1ae093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:49.782313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:49:49 2022] GET /identity/v3/users/85a5af1109c54a2a80385f00893e3137 => generated 247 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:49:49.783509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84baf498-e2f4-4ce2-ab7a-854d4c1ae093 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:49.787764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af052d-3764-4aa0-a0f2-9b179d99122e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-84baf498-e2f4-4ce2-ab7a-854d4c1ae093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:49:49.802119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:49:49.802987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:49:49.803777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:49:49.803777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:49:49.803777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:49:49.803777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:49:49.803989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:49.804063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:49:49.809449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baf498-e2f4-4ce2-ab7a-854d4c1ae093 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:49.873315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:49.896875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af052d-3764-4aa0-a0f2-9b179d99122e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:49.918583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84baf498-e2f4-4ce2-ab7a-854d4c1ae093 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:49.931119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfbd77-0e57-42ee-9844-ccbecd4be171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:49.931783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfbd77-0e57-42ee-9844-ccbecd4be171 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.932055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfbd77-0e57-42ee-9844-ccbecd4be171 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.932405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfbd77-0e57-42ee-9844-ccbecd4be171 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.933585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfbd77-0e57-42ee-9844-ccbecd4be171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:49.935084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfbd77-0e57-42ee-9844-ccbecd4be171 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:49.936966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:49:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:49:49.942615 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfbd77-0e57-42ee-9844-ccbecd4be171 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:49.954565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aaf502-c0d2-48ab-9c86-b13aa36fa41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:49.955225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaf502-c0d2-48ab-9c86-b13aa36fa41e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:49.955506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaf502-c0d2-48ab-9c86-b13aa36fa41e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:49.955783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaf502-c0d2-48ab-9c86-b13aa36fa41e None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:49.956558 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aaf502-c0d2-48ab-9c86-b13aa36fa41e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:49:49.956841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aaf502-c0d2-48ab-9c86-b13aa36fa41e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:49.958268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aaf502-c0d2-48ab-9c86-b13aa36fa41e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:49.975509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 10/20] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:49:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:50.031185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:50.052755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfbd77-0e57-42ee-9844-ccbecd4be171 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:50.091489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ec2bb9-c542-4e45-8355-ca9e35ac4c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:50.092157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec2bb9-c542-4e45-8355-ca9e35ac4c5d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:50.092472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec2bb9-c542-4e45-8355-ca9e35ac4c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:50.092752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ec2bb9-c542-4e45-8355-ca9e35ac4c5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:50.093623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec2bb9-c542-4e45-8355-ca9e35ac4c5d None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.domain.id=default, user.id=85a5af1109c54a2a80385f00893e3137) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:49:50.093929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec2bb9-c542-4e45-8355-ca9e35ac4c5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:50.095529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ec2bb9-c542-4e45-8355-ca9e35ac4c5d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:50.104466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 11/21] 10.222.0.49 () {56 vars in 1322 bytes} [Mon Aug 1 11:49:49 2022] GET /identity/v3/role_assignments?user.id=85a5af1109c54a2a80385f00893e3137&scope.domain.id=default&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:49:52.384492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b763a5-5dc8-466b-a4b5-ff55d1265faa None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:52.385089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b763a5-5dc8-466b-a4b5-ff55d1265faa None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:52.385089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b763a5-5dc8-466b-a4b5-ff55d1265faa None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:52.386661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 11/22] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:52.396546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd2575-e237-4b77-becd-39e51c8e23b2 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:52.397047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd2575-e237-4b77-becd-39e51c8e23b2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:52.397121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd2575-e237-4b77-becd-39e51c8e23b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:52.425951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7bbd2575-e237-4b77-becd-39e51c8e23b2 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:52.427182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7bbd2575-e237-4b77-becd-39e51c8e23b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:52.518690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 12/23] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:52.561871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b763a5-5dc8-466b-a4b5-ff55d1265faa None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:52.633431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a87154-2bcc-457e-b593-45afd43bce43 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:52.633773 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a87154-2bcc-457e-b593-45afd43bce43 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:52.633989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a87154-2bcc-457e-b593-45afd43bce43 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:52.636137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 13/24] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:52.656879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395339e6-5792-4cf4-a36a-215aeae979e8 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:52.657197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395339e6-5792-4cf4-a36a-215aeae979e8 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:52.657434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395339e6-5792-4cf4-a36a-215aeae979e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:52.691842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-395339e6-5792-4cf4-a36a-215aeae979e8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:52.693193 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-395339e6-5792-4cf4-a36a-215aeae979e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:52.769995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b763a5-5dc8-466b-a4b5-ff55d1265faa None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:52.806591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:52.832174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 14/25] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:52.834228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c8ca7-c1ef-48ee-89f1-b911df223f9c None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:52.834508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c8ca7-c1ef-48ee-89f1-b911df223f9c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:52.834778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c8ca7-c1ef-48ee-89f1-b911df223f9c None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:52.837506 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 15/26] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:52.844120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab850980-15f9-4bc3-8990-8b7f9e256111 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:52.844425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab850980-15f9-4bc3-8990-8b7f9e256111 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:52.844719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab850980-15f9-4bc3-8990-8b7f9e256111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:52.849495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b8da9-3b80-4db4-9802-520575cf58e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:52.850242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b8da9-3b80-4db4-9802-520575cf58e9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:52.850525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b8da9-3b80-4db4-9802-520575cf58e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:52.850793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b8da9-3b80-4db4-9802-520575cf58e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:52.862445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:52.875024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b8da9-3b80-4db4-9802-520575cf58e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-7b5b8da9-3b80-4db4-9802-520575cf58e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:49:52.881823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:49:52.882710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:49:52.883438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 12/27] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:49:52.884051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b8da9-3b80-4db4-9802-520575cf58e9 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:52.888402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-ab850980-15f9-4bc3-8990-8b7f9e256111 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:52.889531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ab850980-15f9-4bc3-8990-8b7f9e256111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:52.985383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 16/28] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:52.992903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab850980-15f9-4bc3-8990-8b7f9e256111 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:53.031259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b8da9-3b80-4db4-9802-520575cf58e9 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:53.070183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741174ed-577f-40be-bdde-3a9e6e54e093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:53.070841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741174ed-577f-40be-bdde-3a9e6e54e093 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.071219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741174ed-577f-40be-bdde-3a9e6e54e093 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.071492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741174ed-577f-40be-bdde-3a9e6e54e093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.082936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741174ed-577f-40be-bdde-3a9e6e54e093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:53.084583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741174ed-577f-40be-bdde-3a9e6e54e093 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:53.121172 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab850980-15f9-4bc3-8990-8b7f9e256111 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:53.162096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d9b70-27a9-4ebe-abe9-e977ce1f8ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:53.162680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d9b70-27a9-4ebe-abe9-e977ce1f8ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.162961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d9b70-27a9-4ebe-abe9-e977ce1f8ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.163238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d9b70-27a9-4ebe-abe9-e977ce1f8ad5 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.164004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d9b70-27a9-4ebe-abe9-e977ce1f8ad5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:49:53.164352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d9b70-27a9-4ebe-abe9-e977ce1f8ad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:53.165825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d9b70-27a9-4ebe-abe9-e977ce1f8ad5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:53.167632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:53.176847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 17/29] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:53.183039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b08d0-53da-4791-8d05-a99826f62ec6 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.183327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b08d0-53da-4791-8d05-a99826f62ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.183612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b08d0-53da-4791-8d05-a99826f62ec6 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.185800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 18/30] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:53.187752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b08d0-53da-4791-8d05-a99826f62ec6 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-741174ed-577f-40be-bdde-3a9e6e54e093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:49:53.221389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:49:53.222439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:49:53.223468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:49:53.223468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:49:53.223468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:49:53.223468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:49:53.225639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 13/31] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:49:53.226652 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741174ed-577f-40be-bdde-3a9e6e54e093 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:53.259456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:53.311320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b08d0-53da-4791-8d05-a99826f62ec6 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:53.337705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741174ed-577f-40be-bdde-3a9e6e54e093 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:53.351087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756959d-3f97-460b-84bd-6a07188dda41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:53.351731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756959d-3f97-460b-84bd-6a07188dda41 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.352016 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756959d-3f97-460b-84bd-6a07188dda41 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.352319 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756959d-3f97-460b-84bd-6a07188dda41 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.353204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756959d-3f97-460b-84bd-6a07188dda41 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:53.354807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756959d-3f97-460b-84bd-6a07188dda41 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:53.375701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3143eb-7a35-41b5-aa63-e8011b9559d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:53.376670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3143eb-7a35-41b5-aa63-e8011b9559d0 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.377118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3143eb-7a35-41b5-aa63-e8011b9559d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.377540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3143eb-7a35-41b5-aa63-e8011b9559d0 None admin] PATH_INFO: `/v3/users/85a5af1109c54a2a80385f00893e3137` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.379239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3143eb-7a35-41b5-aa63-e8011b9559d0 None admin] RBAC: Authorizing `identity:get_user(user_id=85a5af1109c54a2a80385f00893e3137)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:53.380932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3143eb-7a35-41b5-aa63-e8011b9559d0 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:53.382732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 14/32] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:49:53 2022] GET /identity/v3/users/85a5af1109c54a2a80385f00893e3137 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:53.384064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceff637-29db-4298-905f-a5db3c791367 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.384583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceff637-29db-4298-905f-a5db3c791367 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.389429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceff637-29db-4298-905f-a5db3c791367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.410929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-eceff637-29db-4298-905f-a5db3c791367 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:53.412510 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-eceff637-29db-4298-905f-a5db3c791367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:53.482269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 15/33] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:53.483427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceff637-29db-4298-905f-a5db3c791367 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:53.513152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:53.513695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 19/34] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Aug 1 11:49:53 2022] POST /identity/v3/roles => generated 219 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:49:53.515546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756959d-3f97-460b-84bd-6a07188dda41 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:53.592238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:53.607474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceff637-29db-4298-905f-a5db3c791367 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:53.645998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756959d-3f97-460b-84bd-6a07188dda41 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:53.650239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ec5f9-49c9-4753-8c2a-b1aeb5be2604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:53.651287 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ec5f9-49c9-4753-8c2a-b1aeb5be2604 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.651762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ec5f9-49c9-4753-8c2a-b1aeb5be2604 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.652250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ec5f9-49c9-4753-8c2a-b1aeb5be2604 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.653225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ec5f9-49c9-4753-8c2a-b1aeb5be2604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:49:53.653788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ec5f9-49c9-4753-8c2a-b1aeb5be2604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:53.655585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ec5f9-49c9-4753-8c2a-b1aeb5be2604 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:53.669962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 16/35] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:53.671137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ec5f9-49c9-4753-8c2a-b1aeb5be2604 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:53.692525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dd496-b83c-4fc0-874a-eae9e3566a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:53.696908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dd496-b83c-4fc0-874a-eae9e3566a1e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.696908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dd496-b83c-4fc0-874a-eae9e3566a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.696908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8dd496-b83c-4fc0-874a-eae9e3566a1e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.696908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dd496-b83c-4fc0-874a-eae9e3566a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:53.696908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8dd496-b83c-4fc0-874a-eae9e3566a1e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:53.698907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 20/36] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:49:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:53.699796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dd496-b83c-4fc0-874a-eae9e3566a1e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:53.793340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:53.814153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ec5f9-49c9-4753-8c2a-b1aeb5be2604 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:53.830387 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8dd496-b83c-4fc0-874a-eae9e3566a1e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:53.856505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:53.859636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987989d2-fd16-45c3-a2d8-f0a3d4882625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:53.860616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987989d2-fd16-45c3-a2d8-f0a3d4882625 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.860897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987989d2-fd16-45c3-a2d8-f0a3d4882625 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.861306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987989d2-fd16-45c3-a2d8-f0a3d4882625 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.862159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987989d2-fd16-45c3-a2d8-f0a3d4882625 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:53.864418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987989d2-fd16-45c3-a2d8-f0a3d4882625 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:53.868715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ee664-b7d8-4c50-bb71-37dab944b7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:53.869385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ee664-b7d8-4c50-bb71-37dab944b7ef None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.869674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ee664-b7d8-4c50-bb71-37dab944b7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.869948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ee664-b7d8-4c50-bb71-37dab944b7ef None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.870756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ee664-b7d8-4c50-bb71-37dab944b7ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:53.872227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13ee664-b7d8-4c50-bb71-37dab944b7ef None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:53.905379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 21/37] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:49:53 2022] POST /identity/v3/projects => generated 301 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:49:53.906300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ee664-b7d8-4c50-bb71-37dab944b7ef None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:53.991129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 17/38] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Aug 1 11:49:53 2022] POST /identity/v3/roles => generated 225 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:49:53.992953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550de45-0330-42a3-9d4b-25ae359a5888 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:53.993389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550de45-0330-42a3-9d4b-25ae359a5888 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:53.993804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550de45-0330-42a3-9d4b-25ae359a5888 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:53.995604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 18/39] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:54.004251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5737e14-cb60-4cfd-9f47-215845fa9f55 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:54.004814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5737e14-cb60-4cfd-9f47-215845fa9f55 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:54.005244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5737e14-cb60-4cfd-9f47-215845fa9f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:54.032027 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-f5737e14-cb60-4cfd-9f47-215845fa9f55 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:54.034193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f5737e14-cb60-4cfd-9f47-215845fa9f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:54.049588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ee664-b7d8-4c50-bb71-37dab944b7ef None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:54.136065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fd99c-2e77-472a-bb29-c335abbec2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:54.137090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fd99c-2e77-472a-bb29-c335abbec2b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:54.137541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fd99c-2e77-472a-bb29-c335abbec2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:54.137996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fd99c-2e77-472a-bb29-c335abbec2b0 None admin] PATH_INFO: `/v3/domains/default/users/85a5af1109c54a2a80385f00893e3137/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:54.141850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fd99c-2e77-472a-bb29-c335abbec2b0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=85a5af1109c54a2a80385f00893e3137, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:54.144821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fd99c-2e77-472a-bb29-c335abbec2b0 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:54.167514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 22/40] 10.222.0.49 () {58 vars in 1361 bytes} [Mon Aug 1 11:49:53 2022] PUT /identity/v3/domains/default/users/85a5af1109c54a2a80385f00893e3137/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:49:54.170473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4d7a7-c3ff-48ca-a151-abc489afde58 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:54.170760 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4d7a7-c3ff-48ca-a151-abc489afde58 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:54.171041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4d7a7-c3ff-48ca-a151-abc489afde58 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:54.172812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 23/41] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:54.179281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1367404-02cd-4663-9fa0-043642989091 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:54.179853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1367404-02cd-4663-9fa0-043642989091 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:54.180437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1367404-02cd-4663-9fa0-043642989091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:54.185959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 19/42] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:54.204799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a1367404-02cd-4663-9fa0-043642989091 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:54.206806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a1367404-02cd-4663-9fa0-043642989091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:54.212540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5737e14-cb60-4cfd-9f47-215845fa9f55 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:54.247834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:54.313856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 24/43] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:54.344993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:54.345195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1367404-02cd-4663-9fa0-043642989091 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:54.401320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5737e14-cb60-4cfd-9f47-215845fa9f55 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:54.438068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e5312-4c34-401f-ae4a-d6f38a990b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:54.438889 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e5312-4c34-401f-ae4a-d6f38a990b5e None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:54.439266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e5312-4c34-401f-ae4a-d6f38a990b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:54.439637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e5312-4c34-401f-ae4a-d6f38a990b5e None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:54.440501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e5312-4c34-401f-ae4a-d6f38a990b5e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:54.442153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e5312-4c34-401f-ae4a-d6f38a990b5e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:54.460092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 20/44] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Aug 1 11:49:54 2022] POST /identity/v3/roles => generated 223 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 11:49:54.468018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1367404-02cd-4663-9fa0-043642989091 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:54.511383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee9801-4ed6-4ccf-b912-10bb3f561465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:54.512023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee9801-4ed6-4ccf-b912-10bb3f561465 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:54.512345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee9801-4ed6-4ccf-b912-10bb3f561465 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:54.512734 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee9801-4ed6-4ccf-b912-10bb3f561465 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:54.514064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee9801-4ed6-4ccf-b912-10bb3f561465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:54.515718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee9801-4ed6-4ccf-b912-10bb3f561465 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:54.517928 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 25/45] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:49:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:54.523892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e5312-4c34-401f-ae4a-d6f38a990b5e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:54.640795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e5312-4c34-401f-ae4a-d6f38a990b5e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:54.676905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c075d-c96d-4f74-bcdb-7cc97fe80216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:54.677569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c075d-c96d-4f74-bcdb-7cc97fe80216 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:54.677848 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c075d-c96d-4f74-bcdb-7cc97fe80216 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:54.678124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c075d-c96d-4f74-bcdb-7cc97fe80216 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:54.678831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c075d-c96d-4f74-bcdb-7cc97fe80216 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:54.680346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c075d-c96d-4f74-bcdb-7cc97fe80216 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:54.686580 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:54.711315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 21/46] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:49:54 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:49:56.488443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173549e7-46e0-4dd0-9df2-55be8cf96867 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:56.489366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173549e7-46e0-4dd0-9df2-55be8cf96867 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:56.489366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173549e7-46e0-4dd0-9df2-55be8cf96867 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:56.490739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 26/47] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:56.496632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088edd48-0ca6-4daa-8405-238526dd28d3 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:56.497330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088edd48-0ca6-4daa-8405-238526dd28d3 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:56.497330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088edd48-0ca6-4daa-8405-238526dd28d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:56.516949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-088edd48-0ca6-4daa-8405-238526dd28d3 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:56.518169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-088edd48-0ca6-4daa-8405-238526dd28d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:56.590149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 22/48] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:56.612752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173549e7-46e0-4dd0-9df2-55be8cf96867 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:56.652269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:56.725634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173549e7-46e0-4dd0-9df2-55be8cf96867 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:56.733380 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cbd7d-cc4d-405e-a490-1282e97c4695 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:56.733675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cbd7d-cc4d-405e-a490-1282e97c4695 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:56.733958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cbd7d-cc4d-405e-a490-1282e97c4695 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:56.735376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 23/49] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:56.741740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b89ae-eb13-4603-b8a6-14e7be961331 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:56.741903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b89ae-eb13-4603-b8a6-14e7be961331 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:56.742220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7b89ae-eb13-4603-b8a6-14e7be961331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:56.762947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-bc7b89ae-eb13-4603-b8a6-14e7be961331 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:56.764197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-bc7b89ae-eb13-4603-b8a6-14e7be961331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:56.768313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407258-9842-4b81-ac0c-e2fb20f2232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:56.768989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407258-9842-4b81-ac0c-e2fb20f2232a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:56.769271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407258-9842-4b81-ac0c-e2fb20f2232a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:56.769555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99407258-9842-4b81-ac0c-e2fb20f2232a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:56.781057 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99407258-9842-4b81-ac0c-e2fb20f2232a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-99407258-9842-4b81-ac0c-e2fb20f2232a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:49:56.786100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:49:56.787026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:49:56.787754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 27/50] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:49:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:49:56.793468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407258-9842-4b81-ac0c-e2fb20f2232a None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:56.835417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 24/51] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:56.858479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b89ae-eb13-4603-b8a6-14e7be961331 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:56.893059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:56.900769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99407258-9842-4b81-ac0c-e2fb20f2232a None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:56.909040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:56.937510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2a12f-8b39-4246-af7b-42a31afbd656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:56.938159 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2a12f-8b39-4246-af7b-42a31afbd656 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:56.938437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2a12f-8b39-4246-af7b-42a31afbd656 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:56.938717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2a12f-8b39-4246-af7b-42a31afbd656 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:56.939461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2a12f-8b39-4246-af7b-42a31afbd656 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:49:56.939773 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2a12f-8b39-4246-af7b-42a31afbd656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:56.941230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2a12f-8b39-4246-af7b-42a31afbd656 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:56.950021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 28/52] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:49:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:56.955496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2a12f-8b39-4246-af7b-42a31afbd656 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:56.961935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7b89ae-eb13-4603-b8a6-14e7be961331 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:56.999807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1274240-8861-4dcd-91ef-87cb7ad5a8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:57.000465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1274240-8861-4dcd-91ef-87cb7ad5a8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:57.000747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1274240-8861-4dcd-91ef-87cb7ad5a8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:57.001025 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1274240-8861-4dcd-91ef-87cb7ad5a8f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:57.002309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1274240-8861-4dcd-91ef-87cb7ad5a8f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:57.003755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1274240-8861-4dcd-91ef-87cb7ad5a8f6 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:57.005548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 25/53] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:49:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:57.011343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1274240-8861-4dcd-91ef-87cb7ad5a8f6 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:57.056459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2a12f-8b39-4246-af7b-42a31afbd656 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:57.097235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61918bf9-8a89-46d9-9377-4c5230397949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:57.098178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61918bf9-8a89-46d9-9377-4c5230397949 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:57.098539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61918bf9-8a89-46d9-9377-4c5230397949 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:57.098894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61918bf9-8a89-46d9-9377-4c5230397949 None admin] PATH_INFO: `/v3/users/85a5af1109c54a2a80385f00893e3137` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:57.100246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61918bf9-8a89-46d9-9377-4c5230397949 None admin] RBAC: Authorizing `identity:get_user(user_id=85a5af1109c54a2a80385f00893e3137)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:57.101737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61918bf9-8a89-46d9-9377-4c5230397949 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:57.103393 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 29/54] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:49:56 2022] GET /identity/v3/users/85a5af1109c54a2a80385f00893e3137 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:49:57.109089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61918bf9-8a89-46d9-9377-4c5230397949 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:57.133562 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1274240-8861-4dcd-91ef-87cb7ad5a8f6 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:57.138997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:57.151545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:57.170668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f3a15-c691-43e0-a41d-50ecdfbf09aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:57.171305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f3a15-c691-43e0-a41d-50ecdfbf09aa None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:57.171604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f3a15-c691-43e0-a41d-50ecdfbf09aa None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:57.171902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f3a15-c691-43e0-a41d-50ecdfbf09aa None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:57.172699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f3a15-c691-43e0-a41d-50ecdfbf09aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:57.174171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597f3a15-c691-43e0-a41d-50ecdfbf09aa None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:57.202183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 26/55] 10.222.0.49 () {60 vars in 1140 bytes} [Mon Aug 1 11:49:57 2022] POST /identity/v3/projects => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:49:57.220857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61918bf9-8a89-46d9-9377-4c5230397949 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:57.261898 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993ddc8-59cd-4e0b-959a-18c22af127c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:57.262499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ddc8-59cd-4e0b-959a-18c22af127c1 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:57.262785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ddc8-59cd-4e0b-959a-18c22af127c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:57.263057 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ddc8-59cd-4e0b-959a-18c22af127c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:57.264475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993ddc8-59cd-4e0b-959a-18c22af127c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:57.265982 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993ddc8-59cd-4e0b-959a-18c22af127c1 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:57.268103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 30/56] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:49:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:57.274531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f3a15-c691-43e0-a41d-50ecdfbf09aa None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:57.391837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597f3a15-c691-43e0-a41d-50ecdfbf09aa None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:57.431170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8100c9d-b721-4716-8481-27161d3aef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:57.431842 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8100c9d-b721-4716-8481-27161d3aef28 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:57.432128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8100c9d-b721-4716-8481-27161d3aef28 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:57.432435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8100c9d-b721-4716-8481-27161d3aef28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:57.433527 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8100c9d-b721-4716-8481-27161d3aef28 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.domain.id=default, user.id=85a5af1109c54a2a80385f00893e3137) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:49:57.433846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8100c9d-b721-4716-8481-27161d3aef28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:57.435341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8100c9d-b721-4716-8481-27161d3aef28 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:57.445675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 27/57] 10.222.0.49 () {56 vars in 1322 bytes} [Mon Aug 1 11:49:57 2022] GET /identity/v3/role_assignments?user.id=85a5af1109c54a2a80385f00893e3137&scope.domain.id=default&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 530 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:59.208977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6dc31-bab4-4231-837e-947d45fd8fed None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:59.209519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6dc31-bab4-4231-837e-947d45fd8fed None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:59.209519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6dc31-bab4-4231-837e-947d45fd8fed None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:59.210807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 31/58] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:49:59.216270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398d86d-0400-42d8-8f10-b93b8589a744 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:59.216689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398d86d-0400-42d8-8f10-b93b8589a744 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:59.216847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398d86d-0400-42d8-8f10-b93b8589a744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:59.236303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a398d86d-0400-42d8-8f10-b93b8589a744 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:49:59.237546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a398d86d-0400-42d8-8f10-b93b8589a744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:49:59.266369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:59.311629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 28/59] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:49:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:49:59.335642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6dc31-bab4-4231-837e-947d45fd8fed None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:59.433422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:49:59.444435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6dc31-bab4-4231-837e-947d45fd8fed None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:59.478136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b0852-3ba5-4b53-91df-f672623678f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:59.478718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0852-3ba5-4b53-91df-f672623678f5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:59.479003 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0852-3ba5-4b53-91df-f672623678f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:59.479280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b0852-3ba5-4b53-91df-f672623678f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:59.480488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b0852-3ba5-4b53-91df-f672623678f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:59.481956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b0852-3ba5-4b53-91df-f672623678f5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:59.483556 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 32/60] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:49:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:49:59.489768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398d86d-0400-42d8-8f10-b93b8589a744 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:49:59.595278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398d86d-0400-42d8-8f10-b93b8589a744 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:49:59.630792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee366-1a64-4c3d-bb9c-33f3094bcb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:49:59.631418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee366-1a64-4c3d-bb9c-33f3094bcb77 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:49:59.631701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee366-1a64-4c3d-bb9c-33f3094bcb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:49:59.631975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee366-1a64-4c3d-bb9c-33f3094bcb77 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:49:59.632707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ee366-1a64-4c3d-bb9c-33f3094bcb77 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:49:59.634181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ee366-1a64-4c3d-bb9c-33f3094bcb77 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:49:59.661322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 29/61] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:49:59 2022] POST /identity/v3/users => generated 275 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:02.781083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068a015-a5ed-47c8-951d-ae3e8b4f9df6 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:02.781599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068a015-a5ed-47c8-951d-ae3e8b4f9df6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:02.781705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068a015-a5ed-47c8-951d-ae3e8b4f9df6 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:02.783014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 33/62] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:02.788609 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca595df-1f78-466c-a482-7e6412701fd4 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:02.789383 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca595df-1f78-466c-a482-7e6412701fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:02.789804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca595df-1f78-466c-a482-7e6412701fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:02.817165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-cca595df-1f78-466c-a482-7e6412701fd4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:02.818649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-cca595df-1f78-466c-a482-7e6412701fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:02.835099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:02.919025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226c546-d42b-406e-ab1f-d3f39b9702e3 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:02.919375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226c546-d42b-406e-ab1f-d3f39b9702e3 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:02.919596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b226c546-d42b-406e-ab1f-d3f39b9702e3 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:02.921137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 34/63] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:02.922344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 30/64] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:02.938548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd834bb-3f46-4304-b8fe-dd2cd2af2646 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:02.938832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd834bb-3f46-4304-b8fe-dd2cd2af2646 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:02.939119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd834bb-3f46-4304-b8fe-dd2cd2af2646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:02.963787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7cd834bb-3f46-4304-b8fe-dd2cd2af2646 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:02.965060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7cd834bb-3f46-4304-b8fe-dd2cd2af2646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:02.973485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca595df-1f78-466c-a482-7e6412701fd4 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:03.077733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 35/65] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:03.078912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7f62a-883f-4091-bc72-83920d3c4083 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.079202 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7f62a-883f-4091-bc72-83920d3c4083 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.079482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7f62a-883f-4091-bc72-83920d3c4083 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.080826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 36/66] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:03.084367 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca595df-1f78-466c-a482-7e6412701fd4 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:03.086401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bb826-544b-4555-83e5-0eed2bc99667 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.086848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bb826-544b-4555-83e5-0eed2bc99667 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.087273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bb826-544b-4555-83e5-0eed2bc99667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.116676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-205bb826-544b-4555-83e5-0eed2bc99667 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:03.118154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-205bb826-544b-4555-83e5-0eed2bc99667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:03.133431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf633fd-060a-4e06-bf39-81595bd9aec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:03.134232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf633fd-060a-4e06-bf39-81595bd9aec6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.134715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf633fd-060a-4e06-bf39-81595bd9aec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.135134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf633fd-060a-4e06-bf39-81595bd9aec6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.146820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf633fd-060a-4e06-bf39-81595bd9aec6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-1cf633fd-060a-4e06-bf39-81595bd9aec6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:03.152824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:03.153922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:03.154883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:03.157046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 31/67] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:03.158207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf633fd-060a-4e06-bf39-81595bd9aec6 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:03.170440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:03.193399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 37/68] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:03.194282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bb826-544b-4555-83e5-0eed2bc99667 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:03.228197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:03.271906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf633fd-060a-4e06-bf39-81595bd9aec6 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:03.309771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76371985-665f-48e6-ac0a-a882f926cb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:03.310461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76371985-665f-48e6-ac0a-a882f926cb32 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.310746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76371985-665f-48e6-ac0a-a882f926cb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.311124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76371985-665f-48e6-ac0a-a882f926cb32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.317529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bb826-544b-4555-83e5-0eed2bc99667 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:03.321808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76371985-665f-48e6-ac0a-a882f926cb32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-76371985-665f-48e6-ac0a-a882f926cb32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:03.326904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:03.327843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:03.328661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:03.332776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 32/69] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:50:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:03.334678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635eb3f-38ad-4292-9858-36e8df9bd123 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.334966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635eb3f-38ad-4292-9858-36e8df9bd123 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.335699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635eb3f-38ad-4292-9858-36e8df9bd123 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.337529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 33/70] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:03.338137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635eb3f-38ad-4292-9858-36e8df9bd123 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:03.363037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a264f2-5cfa-4613-91a7-cdc797025238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:03.364630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a264f2-5cfa-4613-91a7-cdc797025238 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.365090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a264f2-5cfa-4613-91a7-cdc797025238 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.365790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a264f2-5cfa-4613-91a7-cdc797025238 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.367133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a264f2-5cfa-4613-91a7-cdc797025238 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:03.367899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a264f2-5cfa-4613-91a7-cdc797025238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:03.369859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a264f2-5cfa-4613-91a7-cdc797025238 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:03.381237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 38/71] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:03.382402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a264f2-5cfa-4613-91a7-cdc797025238 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:03.413219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:03.464988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635eb3f-38ad-4292-9858-36e8df9bd123 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:03.497401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:03.507359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5c100-2ed9-4ef8-b1a7-d2400ff5de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:03.508115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5c100-2ed9-4ef8-b1a7-d2400ff5de4e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.508431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5c100-2ed9-4ef8-b1a7-d2400ff5de4e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.508703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5c100-2ed9-4ef8-b1a7-d2400ff5de4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.519189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a264f2-5cfa-4613-91a7-cdc797025238 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:03.519966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5c100-2ed9-4ef8-b1a7-d2400ff5de4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-0bd5c100-2ed9-4ef8-b1a7-d2400ff5de4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:03.525074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:03.525895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:03.526626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 34/72] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:03.527792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a96749-c7a0-4e25-a7eb-c56053584ab5 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.527792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a96749-c7a0-4e25-a7eb-c56053584ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.527934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a96749-c7a0-4e25-a7eb-c56053584ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.546464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-00a96749-c7a0-4e25-a7eb-c56053584ab5 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:03.547703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-00a96749-c7a0-4e25-a7eb-c56053584ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:03.558249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d979b06-b2b8-4a3b-994a-dc236ae21ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:03.558877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d979b06-b2b8-4a3b-994a-dc236ae21ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.559157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d979b06-b2b8-4a3b-994a-dc236ae21ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.559432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d979b06-b2b8-4a3b-994a-dc236ae21ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.560150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d979b06-b2b8-4a3b-994a-dc236ae21ca8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:03.560482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d979b06-b2b8-4a3b-994a-dc236ae21ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:03.561866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d979b06-b2b8-4a3b-994a-dc236ae21ca8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:03.569941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 39/73] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:03.573193 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d979b06-b2b8-4a3b-994a-dc236ae21ca8 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:03.615459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 35/74] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:03.616331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a96749-c7a0-4e25-a7eb-c56053584ab5 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:03.673307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:03.688118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d979b06-b2b8-4a3b-994a-dc236ae21ca8 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:03.716030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a96749-c7a0-4e25-a7eb-c56053584ab5 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:03.724735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747f82b-9efd-4abf-bee3-6bf5c43bb837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:03.725632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747f82b-9efd-4abf-bee3-6bf5c43bb837 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.725918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747f82b-9efd-4abf-bee3-6bf5c43bb837 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.726189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747f82b-9efd-4abf-bee3-6bf5c43bb837 None admin] PATH_INFO: `/v3/users/32192a9fb66c412e97d89044a31c5688` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.740186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747f82b-9efd-4abf-bee3-6bf5c43bb837 None admin] RBAC: Authorizing `identity:get_user(user_id=32192a9fb66c412e97d89044a31c5688)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:03.741732 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747f82b-9efd-4abf-bee3-6bf5c43bb837 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:03.747880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 40/75] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:03 2022] GET /identity/v3/users/32192a9fb66c412e97d89044a31c5688 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:03.747880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac513e7b-fd49-4fe3-9846-60392f834a8c None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.747880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac513e7b-fd49-4fe3-9846-60392f834a8c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.747880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac513e7b-fd49-4fe3-9846-60392f834a8c None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.747880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 41/76] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:03.747880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac513e7b-fd49-4fe3-9846-60392f834a8c None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:03.751127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b64e1e-c97e-42e1-85e7-7808eb746c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:03.751291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:03.751812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b64e1e-c97e-42e1-85e7-7808eb746c96 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.752105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b64e1e-c97e-42e1-85e7-7808eb746c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.752754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b64e1e-c97e-42e1-85e7-7808eb746c96 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.753459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b64e1e-c97e-42e1-85e7-7808eb746c96 None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:03.753820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b64e1e-c97e-42e1-85e7-7808eb746c96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:03.755302 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b64e1e-c97e-42e1-85e7-7808eb746c96 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:03.767739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 36/77] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:03.768590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b64e1e-c97e-42e1-85e7-7808eb746c96 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:03.863760 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac513e7b-fd49-4fe3-9846-60392f834a8c None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:03.871737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b64e1e-c97e-42e1-85e7-7808eb746c96 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:03.894213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:03.901994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132c00f-e707-457e-bfce-5a67595df3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:03.902650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132c00f-e707-457e-bfce-5a67595df3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.902932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132c00f-e707-457e-bfce-5a67595df3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.903207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132c00f-e707-457e-bfce-5a67595df3b1 None admin] PATH_INFO: `/v3/users/85a5af1109c54a2a80385f00893e3137` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.904419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132c00f-e707-457e-bfce-5a67595df3b1 None admin] RBAC: Authorizing `identity:get_user(user_id=85a5af1109c54a2a80385f00893e3137)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:03.905907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132c00f-e707-457e-bfce-5a67595df3b1 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:03.907518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 42/78] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:03 2022] GET /identity/v3/users/85a5af1109c54a2a80385f00893e3137 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:03.908281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132c00f-e707-457e-bfce-5a67595df3b1 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:03.909006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c62eb-dd05-4af1-b12c-86df4c0d3fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:03.909612 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c62eb-dd05-4af1-b12c-86df4c0d3fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.909898 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c62eb-dd05-4af1-b12c-86df4c0d3fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.910175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c62eb-dd05-4af1-b12c-86df4c0d3fc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.922023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c62eb-dd05-4af1-b12c-86df4c0d3fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-839c62eb-dd05-4af1-b12c-86df4c0d3fc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:03.926994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:03.927933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:03.928739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 37/79] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:03.932633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d977c-f5c7-4683-9c52-ef89664b2813 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:03.932920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d977c-f5c7-4683-9c52-ef89664b2813 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:03.933217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d977c-f5c7-4683-9c52-ef89664b2813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:03.951498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-965d977c-f5c7-4683-9c52-ef89664b2813 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:03.952782 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-965d977c-f5c7-4683-9c52-ef89664b2813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:03.979028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:04.011323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132c00f-e707-457e-bfce-5a67595df3b1 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:04.023319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 38/80] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:04.024189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d977c-f5c7-4683-9c52-ef89664b2813 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:04.055061 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9bff3-0441-4ffa-b7bc-1cad2605a64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:04.055646 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9bff3-0441-4ffa-b7bc-1cad2605a64e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:04.055939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9bff3-0441-4ffa-b7bc-1cad2605a64e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:04.056312 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9bff3-0441-4ffa-b7bc-1cad2605a64e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:04.125836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9bff3-0441-4ffa-b7bc-1cad2605a64e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:04.127172 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9bff3-0441-4ffa-b7bc-1cad2605a64e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:04.129857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d977c-f5c7-4683-9c52-ef89664b2813 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:04.140420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:04.164961 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a665c-8394-4695-b326-e11830903f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:04.165682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a665c-8394-4695-b326-e11830903f10 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:04.165970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a665c-8394-4695-b326-e11830903f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:04.166246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a665c-8394-4695-b326-e11830903f10 None admin] PATH_INFO: `/v3/users/32192a9fb66c412e97d89044a31c5688` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:04.167444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a665c-8394-4695-b326-e11830903f10 None admin] RBAC: Authorizing `identity:get_user(user_id=32192a9fb66c412e97d89044a31c5688)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:04.168969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a665c-8394-4695-b326-e11830903f10 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:04.170623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 39/81] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:04 2022] GET /identity/v3/users/32192a9fb66c412e97d89044a31c5688 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:04.171371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a665c-8394-4695-b326-e11830903f10 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:04.195272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:04.230623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 43/82] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:04.231478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9bff3-0441-4ffa-b7bc-1cad2605a64e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:04.277087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a665c-8394-4695-b326-e11830903f10 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:04.312313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e7e78-01c2-4fbe-88ba-90310384ca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:04.312952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e7e78-01c2-4fbe-88ba-90310384ca32 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:04.313235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e7e78-01c2-4fbe-88ba-90310384ca32 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:04.313511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e7e78-01c2-4fbe-88ba-90310384ca32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:04.333010 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9bff3-0441-4ffa-b7bc-1cad2605a64e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:04.363739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:04.368658 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18507a11-1863-41c8-8585-b1614a988b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:04.369292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18507a11-1863-41c8-8585-b1614a988b2c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:04.369573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18507a11-1863-41c8-8585-b1614a988b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:04.369848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18507a11-1863-41c8-8585-b1614a988b2c None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:04.370585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18507a11-1863-41c8-8585-b1614a988b2c None admin] RBAC: Adding query filter params (name=member) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:04.370892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18507a11-1863-41c8-8585-b1614a988b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:04.372324 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18507a11-1863-41c8-8585-b1614a988b2c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:04.380940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 44/83] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:04.381620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18507a11-1863-41c8-8585-b1614a988b2c None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:04.389058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e7e78-01c2-4fbe-88ba-90310384ca32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:04.390608 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e7e78-01c2-4fbe-88ba-90310384ca32 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:04.415184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:04.491607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18507a11-1863-41c8-8585-b1614a988b2c None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:04.507919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 40/84] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:04.508818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e7e78-01c2-4fbe-88ba-90310384ca32 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:04.527972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62193bb-33a5-45e8-9756-a18bde4cf3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:04.528769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62193bb-33a5-45e8-9756-a18bde4cf3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:04.528958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62193bb-33a5-45e8-9756-a18bde4cf3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:04.529235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62193bb-33a5-45e8-9756-a18bde4cf3e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:04.530418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62193bb-33a5-45e8-9756-a18bde4cf3e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:04.531928 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62193bb-33a5-45e8-9756-a18bde4cf3e3 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:04.533579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 45/85] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:04.534352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62193bb-33a5-45e8-9756-a18bde4cf3e3 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:04.594163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:04.616231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e7e78-01c2-4fbe-88ba-90310384ca32 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:04.637646 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:04.640995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62193bb-33a5-45e8-9756-a18bde4cf3e3 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:04.651959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf962590-efb4-47a6-bae2-2b8d620770e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:04.652628 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf962590-efb4-47a6-bae2-2b8d620770e7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:04.652910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf962590-efb4-47a6-bae2-2b8d620770e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:04.653188 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf962590-efb4-47a6-bae2-2b8d620770e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:04.681954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1489e1-c7b4-48b3-8e9c-6e9df043cf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:04.682996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1489e1-c7b4-48b3-8e9c-6e9df043cf5e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:04.683448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1489e1-c7b4-48b3-8e9c-6e9df043cf5e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:04.683880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1489e1-c7b4-48b3-8e9c-6e9df043cf5e None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:04.685852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1489e1-c7b4-48b3-8e9c-6e9df043cf5e None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:04.688279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1489e1-c7b4-48b3-8e9c-6e9df043cf5e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:04.691123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 46/86] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:04 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:04.692414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1489e1-c7b4-48b3-8e9c-6e9df043cf5e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:04.739244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf962590-efb4-47a6-bae2-2b8d620770e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:04.741872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf962590-efb4-47a6-bae2-2b8d620770e7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:04.812960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1489e1-c7b4-48b3-8e9c-6e9df043cf5e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:04.827007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:04.850440 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c15ff-1dcd-4546-8d4a-2dc309b03693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:04.851109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c15ff-1dcd-4546-8d4a-2dc309b03693 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:04.851390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c15ff-1dcd-4546-8d4a-2dc309b03693 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:04.851641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 41/87] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:04.851732 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30c15ff-1dcd-4546-8d4a-2dc309b03693 None admin] PATH_INFO: `/v3/users/32192a9fb66c412e97d89044a31c5688` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:04.852535 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf962590-efb4-47a6-bae2-2b8d620770e7 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:04.853085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c15ff-1dcd-4546-8d4a-2dc309b03693 None admin] RBAC: Authorizing `identity:get_user(user_id=32192a9fb66c412e97d89044a31c5688)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:04.854569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30c15ff-1dcd-4546-8d4a-2dc309b03693 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:04.856326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 47/88] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:04 2022] GET /identity/v3/users/32192a9fb66c412e97d89044a31c5688 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:04.856980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c15ff-1dcd-4546-8d4a-2dc309b03693 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:04.894063 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:04.961156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf962590-efb4-47a6-bae2-2b8d620770e7 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:04.961744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30c15ff-1dcd-4546-8d4a-2dc309b03693 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:04.995649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9621921-49d8-4c85-a7d4-2fb05e85ae32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:04.996332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9621921-49d8-4c85-a7d4-2fb05e85ae32 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:04.996619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9621921-49d8-4c85-a7d4-2fb05e85ae32 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:04.996892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9621921-49d8-4c85-a7d4-2fb05e85ae32 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:04.998118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9621921-49d8-4c85-a7d4-2fb05e85ae32 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:04.999567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9621921-49d8-4c85-a7d4-2fb05e85ae32 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:05.001136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437d23a-6516-497c-aaf4-60bdaac0b629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:05.001270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 42/89] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:04 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:05.006231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437d23a-6516-497c-aaf4-60bdaac0b629 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:05.006355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9621921-49d8-4c85-a7d4-2fb05e85ae32 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:05.006428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437d23a-6516-497c-aaf4-60bdaac0b629 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:05.006428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437d23a-6516-497c-aaf4-60bdaac0b629 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:05.006428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4437d23a-6516-497c-aaf4-60bdaac0b629 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:05.006428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4437d23a-6516-497c-aaf4-60bdaac0b629 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:05.033145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 48/90] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:50:04 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:05.033998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437d23a-6516-497c-aaf4-60bdaac0b629 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:05.064062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:05.136902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9621921-49d8-4c85-a7d4-2fb05e85ae32 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:05.154284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:05.166972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437d23a-6516-497c-aaf4-60bdaac0b629 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:05.178586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e1140-d40f-424e-a3ea-969ee801d21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:05.179373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e1140-d40f-424e-a3ea-969ee801d21a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:05.179551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e1140-d40f-424e-a3ea-969ee801d21a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:05.179820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e1140-d40f-424e-a3ea-969ee801d21a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:05.180706 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e1140-d40f-424e-a3ea-969ee801d21a None admin] RBAC: Adding query filter params (role.id=8e13cd96bb124d52b5c8883fe960424f, scope.project.id=183e3860f51e45b8828e2c44923ca1f1, user.id=32192a9fb66c412e97d89044a31c5688) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:05.181056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e1140-d40f-424e-a3ea-969ee801d21a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:05.182555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e1140-d40f-424e-a3ea-969ee801d21a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:05.196329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 43/91] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:05 2022] GET /identity/v3/role_assignments?user.id=32192a9fb66c412e97d89044a31c5688&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=8e13cd96bb124d52b5c8883fe960424f => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:05.196329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e1140-d40f-424e-a3ea-969ee801d21a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:05.219090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97055489-f220-40df-9d7c-94276c3fff7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:05.219816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97055489-f220-40df-9d7c-94276c3fff7f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:05.220740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97055489-f220-40df-9d7c-94276c3fff7f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:05.221039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97055489-f220-40df-9d7c-94276c3fff7f None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:05.223457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97055489-f220-40df-9d7c-94276c3fff7f None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:05.226693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97055489-f220-40df-9d7c-94276c3fff7f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:05.226852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 49/92] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:05 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:05.228761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97055489-f220-40df-9d7c-94276c3fff7f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:05.340020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e1140-d40f-424e-a3ea-969ee801d21a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:05.353786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:05.384266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97055489-f220-40df-9d7c-94276c3fff7f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:05.387515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e4a72-26f8-45bb-bf6e-1db966e5aadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:05.388285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e4a72-26f8-45bb-bf6e-1db966e5aadd None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:05.388602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e4a72-26f8-45bb-bf6e-1db966e5aadd None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:05.388943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e4a72-26f8-45bb-bf6e-1db966e5aadd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:05.433667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30da93-015c-4e2c-aeeb-0e4da3ec7d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:05.434361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30da93-015c-4e2c-aeeb-0e4da3ec7d9e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:05.434640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30da93-015c-4e2c-aeeb-0e4da3ec7d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:05.434914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30da93-015c-4e2c-aeeb-0e4da3ec7d9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:05.435740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30da93-015c-4e2c-aeeb-0e4da3ec7d9e None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=183e3860f51e45b8828e2c44923ca1f1, user.id=85a5af1109c54a2a80385f00893e3137) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:05.436314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30da93-015c-4e2c-aeeb-0e4da3ec7d9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:05.438756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30da93-015c-4e2c-aeeb-0e4da3ec7d9e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:05.445843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 50/93] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:05 2022] GET /identity/v3/role_assignments?user.id=85a5af1109c54a2a80385f00893e3137&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:05.447479 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30da93-015c-4e2c-aeeb-0e4da3ec7d9e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:05.481699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e4a72-26f8-45bb-bf6e-1db966e5aadd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:05.483694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:05.485089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e4a72-26f8-45bb-bf6e-1db966e5aadd None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:05.568635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30da93-015c-4e2c-aeeb-0e4da3ec7d9e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:05.605924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62111e6b-5904-4eb6-b4ec-7a16e9f775e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:05.606560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62111e6b-5904-4eb6-b4ec-7a16e9f775e1 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:05.606840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62111e6b-5904-4eb6-b4ec-7a16e9f775e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:05.607115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62111e6b-5904-4eb6-b4ec-7a16e9f775e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:05.607926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62111e6b-5904-4eb6-b4ec-7a16e9f775e1 None admin] RBAC: Adding query filter params (role.id=4f5140de9df4440a8fb63d93249d14d1, scope.project.id=183e3860f51e45b8828e2c44923ca1f1, user.id=32192a9fb66c412e97d89044a31c5688) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:05.608234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62111e6b-5904-4eb6-b4ec-7a16e9f775e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:05.609753 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62111e6b-5904-4eb6-b4ec-7a16e9f775e1 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:05.617192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 51/94] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:05 2022] GET /identity/v3/role_assignments?user.id=32192a9fb66c412e97d89044a31c5688&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:05.628402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 44/95] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:05.637749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62111e6b-5904-4eb6-b4ec-7a16e9f775e1 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:05.784779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:05.800685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62111e6b-5904-4eb6-b4ec-7a16e9f775e1 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:05.840219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8289e0-0892-4611-8956-7c26d76365d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:05.841277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8289e0-0892-4611-8956-7c26d76365d8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:05.841755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8289e0-0892-4611-8956-7c26d76365d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:05.842226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8289e0-0892-4611-8956-7c26d76365d8 None admin] PATH_INFO: `/v3/projects/1488de510b514c1a937398d753ff15be` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:05.843726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8289e0-0892-4611-8956-7c26d76365d8 None admin] RBAC: Authorizing `identity:get_project(project_id=1488de510b514c1a937398d753ff15be)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:05.845455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8289e0-0892-4611-8956-7c26d76365d8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:05.847754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 52/96] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:05 2022] GET /identity/v3/projects/1488de510b514c1a937398d753ff15be => generated 312 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:05.857753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e4a72-26f8-45bb-bf6e-1db966e5aadd None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:05.871577 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:06.007235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e4a72-26f8-45bb-bf6e-1db966e5aadd None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:06.066811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bc7a6-211a-470e-acdd-1d9d2a16ce1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:06.067372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bc7a6-211a-470e-acdd-1d9d2a16ce1e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:06.067658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bc7a6-211a-470e-acdd-1d9d2a16ce1e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:06.067938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bc7a6-211a-470e-acdd-1d9d2a16ce1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:06.068809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bc7a6-211a-470e-acdd-1d9d2a16ce1e None admin] RBAC: Adding query filter params (role.id=4f5140de9df4440a8fb63d93249d14d1, scope.project.id=1488de510b514c1a937398d753ff15be, user.id=32192a9fb66c412e97d89044a31c5688) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:06.069125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bc7a6-211a-470e-acdd-1d9d2a16ce1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:06.070662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bc7a6-211a-470e-acdd-1d9d2a16ce1e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:06.080942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 45/97] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:05 2022] GET /identity/v3/role_assignments?user.id=32192a9fb66c412e97d89044a31c5688&scope.project.id=1488de510b514c1a937398d753ff15be&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:07.649420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eba75-d084-4dfd-a8f0-6f0f9dca4e98 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:07.650038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eba75-d084-4dfd-a8f0-6f0f9dca4e98 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:07.650038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eba75-d084-4dfd-a8f0-6f0f9dca4e98 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:07.651558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 53/98] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:07.656805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a936694-b977-49eb-a5f2-5ee5d264efb5 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:07.657228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a936694-b977-49eb-a5f2-5ee5d264efb5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:07.657493 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a936694-b977-49eb-a5f2-5ee5d264efb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:07.681411 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-8a936694-b977-49eb-a5f2-5ee5d264efb5 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:07.682704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8a936694-b977-49eb-a5f2-5ee5d264efb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:07.762766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 46/99] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:07.795477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17eba75-d084-4dfd-a8f0-6f0f9dca4e98 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:07.937004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17eba75-d084-4dfd-a8f0-6f0f9dca4e98 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:07.981010 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4e086-7218-4c66-a777-8ab93d2db6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:07.982078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4e086-7218-4c66-a777-8ab93d2db6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:07.982563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4e086-7218-4c66-a777-8ab93d2db6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:07.983060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4e086-7218-4c66-a777-8ab93d2db6b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:07.998880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4e086-7218-4c66-a777-8ab93d2db6b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-c0c4e086-7218-4c66-a777-8ab93d2db6b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:08.004743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:08.005567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:08.006377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:08.008714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 54/100] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:08.014611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a936694-b977-49eb-a5f2-5ee5d264efb5 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:08.022275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:08.165836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a936694-b977-49eb-a5f2-5ee5d264efb5 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:08.213327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5950c7b6-4883-4751-a7b0-18af8157086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:08.213881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950c7b6-4883-4751-a7b0-18af8157086e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:08.214165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950c7b6-4883-4751-a7b0-18af8157086e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:08.214444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5950c7b6-4883-4751-a7b0-18af8157086e None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:08.215180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950c7b6-4883-4751-a7b0-18af8157086e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:08.215486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950c7b6-4883-4751-a7b0-18af8157086e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:08.216925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5950c7b6-4883-4751-a7b0-18af8157086e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:08.232328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 47/101] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:08.237751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4e086-7218-4c66-a777-8ab93d2db6b8 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:08.273777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:08.393068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4e086-7218-4c66-a777-8ab93d2db6b8 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:08.401819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582e2c9-7e0d-4593-b5ae-7a726dd7cab9 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:08.402139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582e2c9-7e0d-4593-b5ae-7a726dd7cab9 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:08.402394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582e2c9-7e0d-4593-b5ae-7a726dd7cab9 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:08.404067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 48/102] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:08.413419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd6daa-7cd1-40c2-afd0-eef6b60d50a0 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:08.413698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd6daa-7cd1-40c2-afd0-eef6b60d50a0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:08.413986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd6daa-7cd1-40c2-afd0-eef6b60d50a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:08.452732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-8acd6daa-7cd1-40c2-afd0-eef6b60d50a0 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:08.453960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8acd6daa-7cd1-40c2-afd0-eef6b60d50a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:08.464082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34881b-271b-4da8-b375-c31de4ebade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:08.465079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34881b-271b-4da8-b375-c31de4ebade0 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:08.465581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34881b-271b-4da8-b375-c31de4ebade0 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:08.466050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34881b-271b-4da8-b375-c31de4ebade0 None admin] PATH_INFO: `/v3/users/32192a9fb66c412e97d89044a31c5688` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:08.472302 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34881b-271b-4da8-b375-c31de4ebade0 None admin] RBAC: Authorizing `identity:get_user(user_id=32192a9fb66c412e97d89044a31c5688)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:08.474086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34881b-271b-4da8-b375-c31de4ebade0 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:08.476011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 55/103] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:08 2022] GET /identity/v3/users/32192a9fb66c412e97d89044a31c5688 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:08.481797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34881b-271b-4da8-b375-c31de4ebade0 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:08.552784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:08.561814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 49/104] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:08.595304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd6daa-7cd1-40c2-afd0-eef6b60d50a0 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:08.651011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:08.693345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34881b-271b-4da8-b375-c31de4ebade0 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:08.721429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd6daa-7cd1-40c2-afd0-eef6b60d50a0 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:08.758127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5a400-3d56-43b8-8e7f-55591fd851ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:08.758795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5a400-3d56-43b8-8e7f-55591fd851ea None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:08.759141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5a400-3d56-43b8-8e7f-55591fd851ea None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:08.759417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5a400-3d56-43b8-8e7f-55591fd851ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:08.770817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5a400-3d56-43b8-8e7f-55591fd851ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-83b5a400-3d56-43b8-8e7f-55591fd851ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:08.776339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:08.777192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:08.777956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 50/105] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:08.779087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea2b94-b9e1-4f8e-bee1-d6f01377a7a5 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:08.779087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea2b94-b9e1-4f8e-bee1-d6f01377a7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:08.779826 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea2b94-b9e1-4f8e-bee1-d6f01377a7a5 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:08.781172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 51/106] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:08.786021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea2b94-b9e1-4f8e-bee1-d6f01377a7a5 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:08.792396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d4ca0-31d1-4dc4-bf52-a9bd88535548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:08.793385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d4ca0-31d1-4dc4-bf52-a9bd88535548 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:08.793877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d4ca0-31d1-4dc4-bf52-a9bd88535548 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:08.794345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d4ca0-31d1-4dc4-bf52-a9bd88535548 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:08.801223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d4ca0-31d1-4dc4-bf52-a9bd88535548 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:08.803053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d4ca0-31d1-4dc4-bf52-a9bd88535548 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:08.805261 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 56/107] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:08 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:08.806733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb77493-5064-491e-9875-473b352a4d28 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:08.807224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb77493-5064-491e-9875-473b352a4d28 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:08.807750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb77493-5064-491e-9875-473b352a4d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:08.861731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7eb77493-5064-491e-9875-473b352a4d28 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:08.864320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7eb77493-5064-491e-9875-473b352a4d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:08.920328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:08.980564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea2b94-b9e1-4f8e-bee1-d6f01377a7a5 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:08.987790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 57/108] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:08.989270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb77493-5064-491e-9875-473b352a4d28 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:09.016873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d4aa8-65bb-42bb-8a1f-c78403f1131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:09.017526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d4aa8-65bb-42bb-8a1f-c78403f1131e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.017807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d4aa8-65bb-42bb-8a1f-c78403f1131e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.018085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d4aa8-65bb-42bb-8a1f-c78403f1131e None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.018812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d4aa8-65bb-42bb-8a1f-c78403f1131e None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:09.019120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d4aa8-65bb-42bb-8a1f-c78403f1131e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:09.020538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d4aa8-65bb-42bb-8a1f-c78403f1131e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:09.029429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 52/109] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:09.030712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e7b49-fc58-465b-807e-2c21dbb4b762 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.030997 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e7b49-fc58-465b-807e-2c21dbb4b762 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.031270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e7b49-fc58-465b-807e-2c21dbb4b762 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.036836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 53/110] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:09.036836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee404486-c45c-4507-aa45-dc5013803bbf None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.036836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee404486-c45c-4507-aa45-dc5013803bbf None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.036836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee404486-c45c-4507-aa45-dc5013803bbf None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.036836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 54/111] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:09.036836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc5038-3553-4508-bcef-b4f71fc0b1e7 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.036836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc5038-3553-4508-bcef-b4f71fc0b1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.037302 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc5038-3553-4508-bcef-b4f71fc0b1e7 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.038162 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 55/112] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:09.038925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc5038-3553-4508-bcef-b4f71fc0b1e7 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:09.081352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:09.104396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb77493-5064-491e-9875-473b352a4d28 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:09.144075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502f0e3-978c-4aa6-8936-6d9aa7d3910f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:09.145137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502f0e3-978c-4aa6-8936-6d9aa7d3910f None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.145420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502f0e3-978c-4aa6-8936-6d9aa7d3910f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.145693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502f0e3-978c-4aa6-8936-6d9aa7d3910f None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1/users/32192a9fb66c412e97d89044a31c5688/roles/8e13cd96bb124d52b5c8883fe960424f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.147973 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502f0e3-978c-4aa6-8936-6d9aa7d3910f None admin] RBAC: Authorizing `identity:create_grant(project_id=183e3860f51e45b8828e2c44923ca1f1, user_id=32192a9fb66c412e97d89044a31c5688, role_id=8e13cd96bb124d52b5c8883fe960424f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:09.149537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502f0e3-978c-4aa6-8936-6d9aa7d3910f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:09.166722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc5038-3553-4508-bcef-b4f71fc0b1e7 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:09.166932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 58/113] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:50:08 2022] PUT /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1/users/32192a9fb66c412e97d89044a31c5688/roles/8e13cd96bb124d52b5c8883fe960424f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:09.167648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502f0e3-978c-4aa6-8936-6d9aa7d3910f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:09.206847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:09.227449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd6e5f-5794-4a9e-a7e5-2b54daf1022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:09.228266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd6e5f-5794-4a9e-a7e5-2b54daf1022c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.228582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd6e5f-5794-4a9e-a7e5-2b54daf1022c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.228861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd6e5f-5794-4a9e-a7e5-2b54daf1022c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.242138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd6e5f-5794-4a9e-a7e5-2b54daf1022c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-e2fd6e5f-5794-4a9e-a7e5-2b54daf1022c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:09.248506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:09.249614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:09.250524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 56/114] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:50:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:09.256450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d509659-b3a4-4e0c-a17a-53b488e8ecb7 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.256739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d509659-b3a4-4e0c-a17a-53b488e8ecb7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.257018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d509659-b3a4-4e0c-a17a-53b488e8ecb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.276573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1d509659-b3a4-4e0c-a17a-53b488e8ecb7 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:09.277818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1d509659-b3a4-4e0c-a17a-53b488e8ecb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:09.292406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502f0e3-978c-4aa6-8936-6d9aa7d3910f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:09.341965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b9d48-35d0-440b-9cac-133804716988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:09.342641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b9d48-35d0-440b-9cac-133804716988 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.342923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b9d48-35d0-440b-9cac-133804716988 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.343198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b9d48-35d0-440b-9cac-133804716988 None admin] PATH_INFO: `/v3/users/32192a9fb66c412e97d89044a31c5688` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.345119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b9d48-35d0-440b-9cac-133804716988 None admin] RBAC: Authorizing `identity:get_user(user_id=32192a9fb66c412e97d89044a31c5688)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:09.346602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b9d48-35d0-440b-9cac-133804716988 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:09.348265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 59/115] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:09 2022] GET /identity/v3/users/32192a9fb66c412e97d89044a31c5688 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:09.349616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509b261-753b-421f-a9ca-8ef14a443946 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.349909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509b261-753b-421f-a9ca-8ef14a443946 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.350185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509b261-753b-421f-a9ca-8ef14a443946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.361912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 57/116] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:09.363543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9478bc-4ae8-4f4a-b324-1075a4e95512 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.363834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9478bc-4ae8-4f4a-b324-1075a4e95512 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.364122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9478bc-4ae8-4f4a-b324-1075a4e95512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.380336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d509b261-753b-421f-a9ca-8ef14a443946 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:09.380336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d509b261-753b-421f-a9ca-8ef14a443946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:09.383257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:09.390958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ed9478bc-4ae8-4f4a-b324-1075a4e95512 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:09.392859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ed9478bc-4ae8-4f4a-b324-1075a4e95512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:09.463009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 60/117] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:09.464023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509b261-753b-421f-a9ca-8ef14a443946 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:09.466902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 58/118] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:09.469224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9478bc-4ae8-4f4a-b324-1075a4e95512 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:09.524015 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:09.584591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509b261-753b-421f-a9ca-8ef14a443946 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:09.591927 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9478bc-4ae8-4f4a-b324-1075a4e95512 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:09.621624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b004696-474e-45a8-a3fa-a4513e68f65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:09.622419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b004696-474e-45a8-a3fa-a4513e68f65d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.622808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b004696-474e-45a8-a3fa-a4513e68f65d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.623161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b004696-474e-45a8-a3fa-a4513e68f65d None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.623994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b004696-474e-45a8-a3fa-a4513e68f65d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:09.624436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b004696-474e-45a8-a3fa-a4513e68f65d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:09.625938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b004696-474e-45a8-a3fa-a4513e68f65d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:09.629619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82125d0-872a-410e-8ba5-41e204a96fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:09.630687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82125d0-872a-410e-8ba5-41e204a96fac None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.631187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82125d0-872a-410e-8ba5-41e204a96fac None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.631662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82125d0-872a-410e-8ba5-41e204a96fac None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.632928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82125d0-872a-410e-8ba5-41e204a96fac None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:09.634450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82125d0-872a-410e-8ba5-41e204a96fac None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:09.639893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 61/119] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:09.639893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b004696-474e-45a8-a3fa-a4513e68f65d None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:09.640140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 59/120] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:09 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:09.640140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82125d0-872a-410e-8ba5-41e204a96fac None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:09.640284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:09.751032 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b004696-474e-45a8-a3fa-a4513e68f65d None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:09.765877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82125d0-872a-410e-8ba5-41e204a96fac None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:09.773452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:09.790547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7e8f9-82e1-4860-87b1-fc36804bcec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:09.791197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7e8f9-82e1-4860-87b1-fc36804bcec9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.791469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7e8f9-82e1-4860-87b1-fc36804bcec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.791746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7e8f9-82e1-4860-87b1-fc36804bcec9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.792927 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7e8f9-82e1-4860-87b1-fc36804bcec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:09.794379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7e8f9-82e1-4860-87b1-fc36804bcec9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:09.795987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 62/121] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:09.796830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7e8f9-82e1-4860-87b1-fc36804bcec9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:09.808206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774ff88-803e-4109-bada-0585b7035c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:09.809118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ff88-803e-4109-bada-0585b7035c2e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.809550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ff88-803e-4109-bada-0585b7035c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.809828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774ff88-803e-4109-bada-0585b7035c2e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.820818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774ff88-803e-4109-bada-0585b7035c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-1774ff88-803e-4109-bada-0585b7035c2e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:09.825902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:09.826737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:09.827531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 60/122] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:09.832799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774ff88-803e-4109-bada-0585b7035c2e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:09.885249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:09.922091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7e8f9-82e1-4860-87b1-fc36804bcec9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:09.949627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774ff88-803e-4109-bada-0585b7035c2e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:09.962794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045a0f2-4e71-4041-a19e-4e027f250c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:09.963454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045a0f2-4e71-4041-a19e-4e027f250c2b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.963736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045a0f2-4e71-4041-a19e-4e027f250c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.964069 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045a0f2-4e71-4041-a19e-4e027f250c2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:09.975930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045a0f2-4e71-4041-a19e-4e027f250c2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-1045a0f2-4e71-4041-a19e-4e027f250c2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:09.981614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:09.982613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:09.983539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:09.990594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 63/123] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:09.991675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045a0f2-4e71-4041-a19e-4e027f250c2b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:09.998505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62321420-8e23-45ee-b78d-827b621f45c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:09.999250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62321420-8e23-45ee-b78d-827b621f45c4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:09.999600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62321420-8e23-45ee-b78d-827b621f45c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:09.999942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62321420-8e23-45ee-b78d-827b621f45c4 None admin] PATH_INFO: `/v3/users/85a5af1109c54a2a80385f00893e3137` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:10.001339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62321420-8e23-45ee-b78d-827b621f45c4 None admin] RBAC: Authorizing `identity:get_user(user_id=85a5af1109c54a2a80385f00893e3137)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:10.002919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62321420-8e23-45ee-b78d-827b621f45c4 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:10.004812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 61/124] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:09 2022] GET /identity/v3/users/85a5af1109c54a2a80385f00893e3137 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:10.010088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62321420-8e23-45ee-b78d-827b621f45c4 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:10.044823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:10.101767 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045a0f2-4e71-4041-a19e-4e027f250c2b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:10.123530 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62321420-8e23-45ee-b78d-827b621f45c4 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:10.131704 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:10.139042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d67cb-5923-4060-9c22-ea8c6a17e73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:10.140081 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d67cb-5923-4060-9c22-ea8c6a17e73e None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:10.140410 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d67cb-5923-4060-9c22-ea8c6a17e73e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:10.140698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d67cb-5923-4060-9c22-ea8c6a17e73e None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1/users/32192a9fb66c412e97d89044a31c5688/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:10.142808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d67cb-5923-4060-9c22-ea8c6a17e73e None admin] RBAC: Authorizing `identity:create_grant(project_id=183e3860f51e45b8828e2c44923ca1f1, user_id=32192a9fb66c412e97d89044a31c5688, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:10.144282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d67cb-5923-4060-9c22-ea8c6a17e73e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:10.158336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 64/125] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:50:09 2022] PUT /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1/users/32192a9fb66c412e97d89044a31c5688/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:10.168379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d67cb-5923-4060-9c22-ea8c6a17e73e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:10.176213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894f115-cc23-4293-b986-e39e431528a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:10.177533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894f115-cc23-4293-b986-e39e431528a1 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:10.177819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894f115-cc23-4293-b986-e39e431528a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:10.178365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894f115-cc23-4293-b986-e39e431528a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:10.179090 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894f115-cc23-4293-b986-e39e431528a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:10.181512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894f115-cc23-4293-b986-e39e431528a1 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:10.220155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 62/126] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:50:10 2022] POST /identity/v3/projects => generated 302 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:10.221513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894f115-cc23-4293-b986-e39e431528a1 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:10.334088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:10.339054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d67cb-5923-4060-9c22-ea8c6a17e73e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:10.349902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894f115-cc23-4293-b986-e39e431528a1 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:10.385550 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b33e9-8536-4024-a9ca-8cfbe0109675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:10.386634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b33e9-8536-4024-a9ca-8cfbe0109675 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:10.387529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b33e9-8536-4024-a9ca-8cfbe0109675 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:10.388017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b33e9-8536-4024-a9ca-8cfbe0109675 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:10.389733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b33e9-8536-4024-a9ca-8cfbe0109675 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:10.390416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b33e9-8536-4024-a9ca-8cfbe0109675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:10.392200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b33e9-8536-4024-a9ca-8cfbe0109675 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:10.393444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14dc80-852e-4dfc-9c31-ec16fcd3fb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:10.394097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14dc80-852e-4dfc-9c31-ec16fcd3fb05 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:10.394491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14dc80-852e-4dfc-9c31-ec16fcd3fb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:10.394772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db14dc80-852e-4dfc-9c31-ec16fcd3fb05 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:10.395518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14dc80-852e-4dfc-9c31-ec16fcd3fb05 None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:10.395829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14dc80-852e-4dfc-9c31-ec16fcd3fb05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:10.397459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db14dc80-852e-4dfc-9c31-ec16fcd3fb05 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:10.405654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 65/127] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:10.406935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b33e9-8536-4024-a9ca-8cfbe0109675 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:10.407370 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 63/128] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:10.413185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14dc80-852e-4dfc-9c31-ec16fcd3fb05 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:10.491257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:10.539476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db14dc80-852e-4dfc-9c31-ec16fcd3fb05 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:10.557403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b33e9-8536-4024-a9ca-8cfbe0109675 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:10.577421 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdf349-4865-4a4e-b8d3-24ed24f33552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:10.578085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdf349-4865-4a4e-b8d3-24ed24f33552 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:10.578366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdf349-4865-4a4e-b8d3-24ed24f33552 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:10.578742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bdf349-4865-4a4e-b8d3-24ed24f33552 None admin] PATH_INFO: `/v3/users/375b4c53a8e94a629964fa9a692859eb` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:10.595457 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bdf349-4865-4a4e-b8d3-24ed24f33552 None admin] RBAC: Authorizing `identity:get_user(user_id=375b4c53a8e94a629964fa9a692859eb)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:10.597163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bdf349-4865-4a4e-b8d3-24ed24f33552 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:10.599362 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 64/129] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:10 2022] GET /identity/v3/users/375b4c53a8e94a629964fa9a692859eb => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:10.600112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b47980-80c4-4664-9625-518951a09a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:10.600247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdf349-4865-4a4e-b8d3-24ed24f33552 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:10.600930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b47980-80c4-4664-9625-518951a09a20 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:10.601279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b47980-80c4-4664-9625-518951a09a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:10.601630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b47980-80c4-4664-9625-518951a09a20 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:10.605164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b47980-80c4-4664-9625-518951a09a20 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:10.606669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b47980-80c4-4664-9625-518951a09a20 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:10.608416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 66/130] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:10 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:10.616685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b47980-80c4-4664-9625-518951a09a20 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:10.618543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:10.730135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b47980-80c4-4664-9625-518951a09a20 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:10.741774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bdf349-4865-4a4e-b8d3-24ed24f33552 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:10.752466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:10.769599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3731a69-6a05-4230-8edc-3c8f46f217ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:10.770246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3731a69-6a05-4230-8edc-3c8f46f217ff None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:10.770524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3731a69-6a05-4230-8edc-3c8f46f217ff None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:10.770798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3731a69-6a05-4230-8edc-3c8f46f217ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:10.787326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e1da2-1219-4d28-a67a-00f7f21d219b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:10.792341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e1da2-1219-4d28-a67a-00f7f21d219b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:10.792341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e1da2-1219-4d28-a67a-00f7f21d219b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:10.792341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e1da2-1219-4d28-a67a-00f7f21d219b None admin] PATH_INFO: `/v3/users/32192a9fb66c412e97d89044a31c5688` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:10.794538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e1da2-1219-4d28-a67a-00f7f21d219b None admin] RBAC: Authorizing `identity:get_user(user_id=32192a9fb66c412e97d89044a31c5688)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:10.796003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e1da2-1219-4d28-a67a-00f7f21d219b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:10.797693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 65/131] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:10 2022] GET /identity/v3/users/32192a9fb66c412e97d89044a31c5688 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:10.798441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e1da2-1219-4d28-a67a-00f7f21d219b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:10.900197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3731a69-6a05-4230-8edc-3c8f46f217ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:10.901728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3731a69-6a05-4230-8edc-3c8f46f217ff None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:10.954474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:11.005400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e1da2-1219-4d28-a67a-00f7f21d219b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:11.059868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 67/132] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:11.060053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541e81b-59e0-4d52-95e3-82c225b4eaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:11.060708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541e81b-59e0-4d52-95e3-82c225b4eaae None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:11.060996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541e81b-59e0-4d52-95e3-82c225b4eaae None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:11.061405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541e81b-59e0-4d52-95e3-82c225b4eaae None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1/users/85a5af1109c54a2a80385f00893e3137/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:11.061826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3731a69-6a05-4230-8edc-3c8f46f217ff None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:11.067238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541e81b-59e0-4d52-95e3-82c225b4eaae None admin] RBAC: Authorizing `identity:create_grant(project_id=183e3860f51e45b8828e2c44923ca1f1, user_id=85a5af1109c54a2a80385f00893e3137, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:11.067238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541e81b-59e0-4d52-95e3-82c225b4eaae None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:11.085783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 66/133] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:50:10 2022] PUT /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1/users/85a5af1109c54a2a80385f00893e3137/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:11.086865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541e81b-59e0-4d52-95e3-82c225b4eaae None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:11.115629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:11.240327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541e81b-59e0-4d52-95e3-82c225b4eaae None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:11.265879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3731a69-6a05-4230-8edc-3c8f46f217ff None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:11.280793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287ae3c-df7c-4f34-995b-af74e5aa07ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:11.280793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287ae3c-df7c-4f34-995b-af74e5aa07ba None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:11.280793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287ae3c-df7c-4f34-995b-af74e5aa07ba None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:11.280793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287ae3c-df7c-4f34-995b-af74e5aa07ba None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:11.281192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287ae3c-df7c-4f34-995b-af74e5aa07ba None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:11.281192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4287ae3c-df7c-4f34-995b-af74e5aa07ba None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:11.282856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 67/134] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:11 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:11.289016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287ae3c-df7c-4f34-995b-af74e5aa07ba None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:11.292128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:11.332043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ef12e-7b8c-4c7f-9e00-78f40b65e8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:11.332742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ef12e-7b8c-4c7f-9e00-78f40b65e8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:11.333022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ef12e-7b8c-4c7f-9e00-78f40b65e8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:11.333296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ef12e-7b8c-4c7f-9e00-78f40b65e8c4 None admin] PATH_INFO: `/v3/projects/1488de510b514c1a937398d753ff15be` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:11.334540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ef12e-7b8c-4c7f-9e00-78f40b65e8c4 None admin] RBAC: Authorizing `identity:get_project(project_id=1488de510b514c1a937398d753ff15be)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:11.336035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ef12e-7b8c-4c7f-9e00-78f40b65e8c4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:11.346484 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 68/135] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:11 2022] GET /identity/v3/projects/1488de510b514c1a937398d753ff15be => generated 312 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:11.366109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ef12e-7b8c-4c7f-9e00-78f40b65e8c4 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:11.410258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287ae3c-df7c-4f34-995b-af74e5aa07ba None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:11.459626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e334b63-db6a-4029-b690-916a0fdcd5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:11.460268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e334b63-db6a-4029-b690-916a0fdcd5ab None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:11.460581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e334b63-db6a-4029-b690-916a0fdcd5ab None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:11.460857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e334b63-db6a-4029-b690-916a0fdcd5ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:11.461670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e334b63-db6a-4029-b690-916a0fdcd5ab None admin] RBAC: Adding query filter params (role.id=cc9d3730bd5c4edcb1897fc5de4e031d, scope.project.id=183e3860f51e45b8828e2c44923ca1f1, user.id=375b4c53a8e94a629964fa9a692859eb) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:11.462054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e334b63-db6a-4029-b690-916a0fdcd5ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:11.463539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e334b63-db6a-4029-b690-916a0fdcd5ab None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:11.478943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 68/136] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:11 2022] GET /identity/v3/role_assignments?user.id=375b4c53a8e94a629964fa9a692859eb&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=cc9d3730bd5c4edcb1897fc5de4e031d => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:11.480632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f38292-fa5f-4f0c-8e23-486a58607dea None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:11.481118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f38292-fa5f-4f0c-8e23-486a58607dea None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:11.481589 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f38292-fa5f-4f0c-8e23-486a58607dea None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:11.483173 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 69/137] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:11.489290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760595e5-f629-43f1-93a8-9340498a52b9 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:11.490110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760595e5-f629-43f1-93a8-9340498a52b9 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:11.490582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760595e5-f629-43f1-93a8-9340498a52b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:11.515658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-760595e5-f629-43f1-93a8-9340498a52b9 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:11.518045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-760595e5-f629-43f1-93a8-9340498a52b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:11.561368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:11.590284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ef12e-7b8c-4c7f-9e00-78f40b65e8c4 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:11.637750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:11.660882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 70/138] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:11.678949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4d302-0b15-4d1a-8795-98b1f374f9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:11.679711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4d302-0b15-4d1a-8795-98b1f374f9f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:11.680001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4d302-0b15-4d1a-8795-98b1f374f9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:11.680279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4d302-0b15-4d1a-8795-98b1f374f9f1 None admin] PATH_INFO: `/v3/projects/1488de510b514c1a937398d753ff15be/users/32192a9fb66c412e97d89044a31c5688/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:11.686124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4d302-0b15-4d1a-8795-98b1f374f9f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1488de510b514c1a937398d753ff15be, user_id=32192a9fb66c412e97d89044a31c5688, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:11.687688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4d302-0b15-4d1a-8795-98b1f374f9f1 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:11.711379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 69/139] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:50:11 2022] PUT /identity/v3/projects/1488de510b514c1a937398d753ff15be/users/32192a9fb66c412e97d89044a31c5688/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:11.711914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760595e5-f629-43f1-93a8-9340498a52b9 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:11.857359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760595e5-f629-43f1-93a8-9340498a52b9 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:11.902265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b9060-5115-435b-9091-920bff868936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:11.902856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b9060-5115-435b-9091-920bff868936 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:11.903136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b9060-5115-435b-9091-920bff868936 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:11.903410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b9060-5115-435b-9091-920bff868936 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:11.919518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129b9060-5115-435b-9091-920bff868936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-129b9060-5115-435b-9091-920bff868936 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:11.925683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:11.926600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:11.927345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 71/140] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:11.933739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4d302-0b15-4d1a-8795-98b1f374f9f1 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:12.083768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4d302-0b15-4d1a-8795-98b1f374f9f1 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:12.131094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03e979-d48d-4967-a559-67d7ebec64de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:12.131727 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03e979-d48d-4967-a559-67d7ebec64de None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:12.132007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03e979-d48d-4967-a559-67d7ebec64de None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:12.132281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03e979-d48d-4967-a559-67d7ebec64de None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:12.133033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03e979-d48d-4967-a559-67d7ebec64de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:12.133341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03e979-d48d-4967-a559-67d7ebec64de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:12.134855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03e979-d48d-4967-a559-67d7ebec64de None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:12.144218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 70/141] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:12.153488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b9060-5115-435b-9091-920bff868936 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:12.232177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:12.288721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b9060-5115-435b-9091-920bff868936 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:12.332034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e3b2f-f4c7-4fb9-ab47-aa978d8fd3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:12.332720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3b2f-f4c7-4fb9-ab47-aa978d8fd3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:12.333001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3b2f-f4c7-4fb9-ab47-aa978d8fd3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:12.333272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e3b2f-f4c7-4fb9-ab47-aa978d8fd3e5 None admin] PATH_INFO: `/v3/users/32192a9fb66c412e97d89044a31c5688` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:12.336009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e3b2f-f4c7-4fb9-ab47-aa978d8fd3e5 None admin] RBAC: Authorizing `identity:get_user(user_id=32192a9fb66c412e97d89044a31c5688)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:12.337657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e3b2f-f4c7-4fb9-ab47-aa978d8fd3e5 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:12.339315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 72/142] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:12 2022] GET /identity/v3/users/32192a9fb66c412e97d89044a31c5688 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:12.345065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03e979-d48d-4967-a559-67d7ebec64de None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:12.367608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:12.467319 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03e979-d48d-4967-a559-67d7ebec64de None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:12.502343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a66402-2cab-4e00-90ff-fdc091462a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:12.503236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a66402-2cab-4e00-90ff-fdc091462a20 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:12.503681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a66402-2cab-4e00-90ff-fdc091462a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:12.504091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a66402-2cab-4e00-90ff-fdc091462a20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:12.598467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a66402-2cab-4e00-90ff-fdc091462a20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:12.599964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a66402-2cab-4e00-90ff-fdc091462a20 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:12.616389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:12.721368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 71/143] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:12.726781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e3b2f-f4c7-4fb9-ab47-aa978d8fd3e5 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:12.862797 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e3b2f-f4c7-4fb9-ab47-aa978d8fd3e5 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:12.895869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:12.907689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c385746-96a1-4e0c-a970-ea2f61c62ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:12.908375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c385746-96a1-4e0c-a970-ea2f61c62ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:12.908674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c385746-96a1-4e0c-a970-ea2f61c62ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:12.908947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c385746-96a1-4e0c-a970-ea2f61c62ce7 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:12.910599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c385746-96a1-4e0c-a970-ea2f61c62ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:12.912054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c385746-96a1-4e0c-a970-ea2f61c62ce7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:12.914015 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 73/144] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:12 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:12.919778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a66402-2cab-4e00-90ff-fdc091462a20 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:13.035533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a66402-2cab-4e00-90ff-fdc091462a20 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:13.047108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:13.070528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439eff62-39bd-4512-9aaa-fc23e2f0f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:13.071194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439eff62-39bd-4512-9aaa-fc23e2f0f672 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:13.071595 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439eff62-39bd-4512-9aaa-fc23e2f0f672 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:13.071989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439eff62-39bd-4512-9aaa-fc23e2f0f672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:13.073042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439eff62-39bd-4512-9aaa-fc23e2f0f672 None admin] RBAC: Adding query filter params (role.id=8e13cd96bb124d52b5c8883fe960424f, scope.project.id=183e3860f51e45b8828e2c44923ca1f1, user.id=32192a9fb66c412e97d89044a31c5688) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:13.073356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439eff62-39bd-4512-9aaa-fc23e2f0f672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:13.074879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439eff62-39bd-4512-9aaa-fc23e2f0f672 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:13.082721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6326eb-7373-4253-97ed-0d7e491a0b49 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:13.082995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6326eb-7373-4253-97ed-0d7e491a0b49 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:13.083266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6326eb-7373-4253-97ed-0d7e491a0b49 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:13.085077 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 74/145] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:13.085206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 72/146] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:12 2022] GET /identity/v3/role_assignments?user.id=32192a9fb66c412e97d89044a31c5688&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=8e13cd96bb124d52b5c8883fe960424f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:13.099627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf281f1-e2e2-405c-a070-5e6125518703 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:13.099627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf281f1-e2e2-405c-a070-5e6125518703 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:13.100579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf281f1-e2e2-405c-a070-5e6125518703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:13.133380 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-baf281f1-e2e2-405c-a070-5e6125518703 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:13.134718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-baf281f1-e2e2-405c-a070-5e6125518703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:13.232191 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5311c5-8a66-42e8-a93b-707d641bfd93 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:13.232546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5311c5-8a66-42e8-a93b-707d641bfd93 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:13.232782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5311c5-8a66-42e8-a93b-707d641bfd93 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:13.234348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 73/147] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:13.235650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 75/148] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:13.241543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a595-fb92-41c4-bea2-2f2fbeead535 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:13.241830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a595-fb92-41c4-bea2-2f2fbeead535 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:13.242100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426a595-fb92-41c4-bea2-2f2fbeead535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:13.266668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2426a595-fb92-41c4-bea2-2f2fbeead535 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:13.267905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2426a595-fb92-41c4-bea2-2f2fbeead535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:13.273825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf281f1-e2e2-405c-a070-5e6125518703 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:13.404210 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 74/149] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:13.428040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426a595-fb92-41c4-bea2-2f2fbeead535 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:13.493329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf281f1-e2e2-405c-a070-5e6125518703 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:13.545930 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:13.571500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426a595-fb92-41c4-bea2-2f2fbeead535 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:13.574053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:13.575463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197f1ec-a05e-4e8a-b329-0d2be7027950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:13.576465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197f1ec-a05e-4e8a-b329-0d2be7027950 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:13.576902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197f1ec-a05e-4e8a-b329-0d2be7027950 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:13.577324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d197f1ec-a05e-4e8a-b329-0d2be7027950 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:13.606891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d197f1ec-a05e-4e8a-b329-0d2be7027950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-d197f1ec-a05e-4e8a-b329-0d2be7027950 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:13.612323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:13.613812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:13.614766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:13.616682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e76f-46c3-4455-bd4c-128856776ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:13.616921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 76/150] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:13.618922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e76f-46c3-4455-bd4c-128856776ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:13.618922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e76f-46c3-4455-bd4c-128856776ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:13.618922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e76f-46c3-4455-bd4c-128856776ae9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:13.619171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742e76f-46c3-4455-bd4c-128856776ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:13.620644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742e76f-46c3-4455-bd4c-128856776ae9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:13.622249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 75/151] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:13.623461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197f1ec-a05e-4e8a-b329-0d2be7027950 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:13.628794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e76f-46c3-4455-bd4c-128856776ae9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:13.793419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d197f1ec-a05e-4e8a-b329-0d2be7027950 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:13.814313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e76f-46c3-4455-bd4c-128856776ae9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:13.839791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9513c-2817-4ead-a838-15794504ca32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:13.840470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9513c-2817-4ead-a838-15794504ca32 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:13.840750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9513c-2817-4ead-a838-15794504ca32 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:13.841023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9513c-2817-4ead-a838-15794504ca32 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:13.841769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9513c-2817-4ead-a838-15794504ca32 None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:13.842067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9513c-2817-4ead-a838-15794504ca32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:13.843468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9513c-2817-4ead-a838-15794504ca32 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:13.862140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 77/152] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:13.872349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9513c-2817-4ead-a838-15794504ca32 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:13.887380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825585f-b713-4e1a-806b-f96d0b7328c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:13.888505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825585f-b713-4e1a-806b-f96d0b7328c7 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:13.888992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825585f-b713-4e1a-806b-f96d0b7328c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:13.889500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825585f-b713-4e1a-806b-f96d0b7328c7 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:13.890572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825585f-b713-4e1a-806b-f96d0b7328c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:13.893422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825585f-b713-4e1a-806b-f96d0b7328c7 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:13.928852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 76/153] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:50:13 2022] POST /identity/v3/users => generated 283 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 11:50:13.968877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:14.036274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9513c-2817-4ead-a838-15794504ca32 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:14.073874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8db2e0-81bb-4d9d-b337-9fbfa5134bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:14.074790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8db2e0-81bb-4d9d-b337-9fbfa5134bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:14.075216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8db2e0-81bb-4d9d-b337-9fbfa5134bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:14.075638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8db2e0-81bb-4d9d-b337-9fbfa5134bd3 None admin] PATH_INFO: `/v3/users/32192a9fb66c412e97d89044a31c5688` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:14.077032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8db2e0-81bb-4d9d-b337-9fbfa5134bd3 None admin] RBAC: Authorizing `identity:get_user(user_id=32192a9fb66c412e97d89044a31c5688)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:14.078644 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8db2e0-81bb-4d9d-b337-9fbfa5134bd3 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:14.080541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 78/154] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:13 2022] GET /identity/v3/users/32192a9fb66c412e97d89044a31c5688 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:14.089790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825585f-b713-4e1a-806b-f96d0b7328c7 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:14.145555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:14.247083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825585f-b713-4e1a-806b-f96d0b7328c7 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:14.283733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfe7f7-c426-4e68-be13-1ab01283ebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:14.284783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfe7f7-c426-4e68-be13-1ab01283ebae None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:14.285270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfe7f7-c426-4e68-be13-1ab01283ebae None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:14.285748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfe7f7-c426-4e68-be13-1ab01283ebae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:14.329766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f9d0-0b2b-4571-b896-6d7252a71caa None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:14.329948 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f9d0-0b2b-4571-b896-6d7252a71caa None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:14.330229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d822f9d0-0b2b-4571-b896-6d7252a71caa None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:14.331615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 79/155] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:14.338509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f783-c200-4dab-b4d6-564aa8705cd4 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:14.338810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f783-c200-4dab-b4d6-564aa8705cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:14.339083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68f783-c200-4dab-b4d6-564aa8705cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:14.358056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-2b68f783-c200-4dab-b4d6-564aa8705cd4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:14.359290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2b68f783-c200-4dab-b4d6-564aa8705cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:14.365052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfe7f7-c426-4e68-be13-1ab01283ebae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:14.366845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfe7f7-c426-4e68-be13-1ab01283ebae None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:14.409745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:14.454831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 80/156] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:14.477560 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68f783-c200-4dab-b4d6-564aa8705cd4 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:14.503004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 77/157] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:14.509996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfe7f7-c426-4e68-be13-1ab01283ebae None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:14.537505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:14.608467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68f783-c200-4dab-b4d6-564aa8705cd4 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:14.621505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfe7f7-c426-4e68-be13-1ab01283ebae None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:14.647378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b990f3d6-8e28-4c59-8a05-95a479586e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:14.648106 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990f3d6-8e28-4c59-8a05-95a479586e6d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:14.648437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990f3d6-8e28-4c59-8a05-95a479586e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:14.648766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990f3d6-8e28-4c59-8a05-95a479586e6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:14.653509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:14.662974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b990f3d6-8e28-4c59-8a05-95a479586e6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:14.663347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd45ab-7361-4e1c-87b1-994d1f09b763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:14.664020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd45ab-7361-4e1c-87b1-994d1f09b763 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:14.664316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd45ab-7361-4e1c-87b1-994d1f09b763 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:14.664584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd45ab-7361-4e1c-87b1-994d1f09b763 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:14.665853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd45ab-7361-4e1c-87b1-994d1f09b763 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:14.667328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd45ab-7361-4e1c-87b1-994d1f09b763 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-b990f3d6-8e28-4c59-8a05-95a479586e6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:14.667447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:14.668414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:14.669180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 81/158] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:50:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:14.669810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 78/159] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:14 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:14.671691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6811f9e-e968-42a2-982f-2ee56547622e None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:14.676314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6811f9e-e968-42a2-982f-2ee56547622e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:14.676314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6811f9e-e968-42a2-982f-2ee56547622e None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:14.676314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 82/160] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:14.681508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f6f8f-ff08-4218-9f27-845ea139a533 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:14.682110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f6f8f-ff08-4218-9f27-845ea139a533 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:14.682832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f6f8f-ff08-4218-9f27-845ea139a533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:14.683155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6811f9e-e968-42a2-982f-2ee56547622e None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:14.704038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-0a0f6f8f-ff08-4218-9f27-845ea139a533 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:14.705636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0a0f6f8f-ff08-4218-9f27-845ea139a533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:14.779609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 79/161] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:14.785531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f6f8f-ff08-4218-9f27-845ea139a533 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:14.799805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6811f9e-e968-42a2-982f-2ee56547622e None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:14.812688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:14.840619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497fb55-bb8a-4083-9e1b-55081d9926ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:14.841318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497fb55-bb8a-4083-9e1b-55081d9926ae None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:14.841608 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497fb55-bb8a-4083-9e1b-55081d9926ae None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:14.842333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b497fb55-bb8a-4083-9e1b-55081d9926ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:14.843350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497fb55-bb8a-4083-9e1b-55081d9926ae None admin] RBAC: Adding query filter params (role.id=4f5140de9df4440a8fb63d93249d14d1, scope.project.id=183e3860f51e45b8828e2c44923ca1f1, user.id=32192a9fb66c412e97d89044a31c5688) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:14.843756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497fb55-bb8a-4083-9e1b-55081d9926ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:14.847247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b497fb55-bb8a-4083-9e1b-55081d9926ae None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:14.853827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 83/162] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:14 2022] GET /identity/v3/role_assignments?user.id=32192a9fb66c412e97d89044a31c5688&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:14.859039 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497fb55-bb8a-4083-9e1b-55081d9926ae None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:14.961105 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f6f8f-ff08-4218-9f27-845ea139a533 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:14.998011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:15.009711 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b497fb55-bb8a-4083-9e1b-55081d9926ae None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:15.012854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087c2ab-9e91-441d-bd6b-d5686f1764ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:15.013775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087c2ab-9e91-441d-bd6b-d5686f1764ce None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:15.014260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087c2ab-9e91-441d-bd6b-d5686f1764ce None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:15.014724 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087c2ab-9e91-441d-bd6b-d5686f1764ce None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:15.015677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087c2ab-9e91-441d-bd6b-d5686f1764ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:15.016184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087c2ab-9e91-441d-bd6b-d5686f1764ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:15.018540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087c2ab-9e91-441d-bd6b-d5686f1764ce None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:15.031277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 80/163] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:15.033079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31497d4e-2b96-4908-abfd-ccdd449d4fd3 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:15.033616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31497d4e-2b96-4908-abfd-ccdd449d4fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:15.034084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31497d4e-2b96-4908-abfd-ccdd449d4fd3 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:15.036183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 81/164] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:15.038760 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31497d4e-2b96-4908-abfd-ccdd449d4fd3 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:15.066948 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6a024-4305-4311-b454-8509fdeb3d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:15.067698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a024-4305-4311-b454-8509fdeb3d97 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:15.067982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a024-4305-4311-b454-8509fdeb3d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:15.068647 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6a024-4305-4311-b454-8509fdeb3d97 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:15.084853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6a024-4305-4311-b454-8509fdeb3d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-69e6a024-4305-4311-b454-8509fdeb3d97 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:15.092429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:15.093526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:15.094284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 84/165] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:15.095794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59619906-1d12-476b-ae27-35721317ab02 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:15.095794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59619906-1d12-476b-ae27-35721317ab02 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:15.095951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59619906-1d12-476b-ae27-35721317ab02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:15.120423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-59619906-1d12-476b-ae27-35721317ab02 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:15.121613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-59619906-1d12-476b-ae27-35721317ab02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:15.150841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:15.201902 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31497d4e-2b96-4908-abfd-ccdd449d4fd3 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:15.218517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 85/166] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:15.225256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59619906-1d12-476b-ae27-35721317ab02 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:15.298238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302438bd-41a4-4e5c-80ce-6c4c92ab25bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:15.299676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302438bd-41a4-4e5c-80ce-6c4c92ab25bb None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:15.299955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302438bd-41a4-4e5c-80ce-6c4c92ab25bb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:15.300238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302438bd-41a4-4e5c-80ce-6c4c92ab25bb None admin] PATH_INFO: `/v3/users/85a5af1109c54a2a80385f00893e3137` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:15.302136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302438bd-41a4-4e5c-80ce-6c4c92ab25bb None admin] RBAC: Authorizing `identity:get_user(user_id=85a5af1109c54a2a80385f00893e3137)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:15.303835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302438bd-41a4-4e5c-80ce-6c4c92ab25bb None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:15.305595 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 82/167] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:15 2022] GET /identity/v3/users/85a5af1109c54a2a80385f00893e3137 => generated 247 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:15.309317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302438bd-41a4-4e5c-80ce-6c4c92ab25bb None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:15.371345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59619906-1d12-476b-ae27-35721317ab02 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:15.403245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:15.445066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6239bf-74cc-41cd-b3c1-430a778f99ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:15.445731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6239bf-74cc-41cd-b3c1-430a778f99ea None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:15.446088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6239bf-74cc-41cd-b3c1-430a778f99ea None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:15.446367 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6239bf-74cc-41cd-b3c1-430a778f99ea None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:15.447120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6239bf-74cc-41cd-b3c1-430a778f99ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:15.447423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6239bf-74cc-41cd-b3c1-430a778f99ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:15.448883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6239bf-74cc-41cd-b3c1-430a778f99ea None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:15.450285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302438bd-41a4-4e5c-80ce-6c4c92ab25bb None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:15.464712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 86/168] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:15.472660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6239bf-74cc-41cd-b3c1-430a778f99ea None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:15.486682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae32808-ce8c-4c65-b859-32fe91ac8f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:15.487408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32808-ce8c-4c65-b859-32fe91ac8f23 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:15.487693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32808-ce8c-4c65-b859-32fe91ac8f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:15.488363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae32808-ce8c-4c65-b859-32fe91ac8f23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:15.500014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae32808-ce8c-4c65-b859-32fe91ac8f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-5ae32808-ce8c-4c65-b859-32fe91ac8f23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:15.505791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:15.506788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:15.507574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 83/169] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:15.508224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae32808-ce8c-4c65-b859-32fe91ac8f23 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:15.515426 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:15.624798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6239bf-74cc-41cd-b3c1-430a778f99ea None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:15.645085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae32808-ce8c-4c65-b859-32fe91ac8f23 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:15.684059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feda19c-38c6-45ae-a8c8-64fa790b2985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:15.684721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feda19c-38c6-45ae-a8c8-64fa790b2985 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:15.684999 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feda19c-38c6-45ae-a8c8-64fa790b2985 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:15.685270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feda19c-38c6-45ae-a8c8-64fa790b2985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:15.692631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:15.710834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe44276-b1c2-4fc0-9460-e37ee21afce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:15.711529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe44276-b1c2-4fc0-9460-e37ee21afce3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:15.713328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe44276-b1c2-4fc0-9460-e37ee21afce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:15.713606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe44276-b1c2-4fc0-9460-e37ee21afce3 None admin] PATH_INFO: `/v3/users/375b4c53a8e94a629964fa9a692859eb` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:15.714874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe44276-b1c2-4fc0-9460-e37ee21afce3 None admin] RBAC: Authorizing `identity:get_user(user_id=375b4c53a8e94a629964fa9a692859eb)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:15.717438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe44276-b1c2-4fc0-9460-e37ee21afce3 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:15.722316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 84/170] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:15 2022] GET /identity/v3/users/375b4c53a8e94a629964fa9a692859eb => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:15.723142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe44276-b1c2-4fc0-9460-e37ee21afce3 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:15.826526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feda19c-38c6-45ae-a8c8-64fa790b2985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:15.831342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feda19c-38c6-45ae-a8c8-64fa790b2985 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:15.870820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:15.922975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe44276-b1c2-4fc0-9460-e37ee21afce3 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:15.993304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9b6cb-c497-4108-8c16-9e1e826bf871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:15.993940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9b6cb-c497-4108-8c16-9e1e826bf871 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:15.994220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9b6cb-c497-4108-8c16-9e1e826bf871 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:15.994491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9b6cb-c497-4108-8c16-9e1e826bf871 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:15.995217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9b6cb-c497-4108-8c16-9e1e826bf871 None admin] RBAC: Adding query filter params (name=member) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:15.995521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9b6cb-c497-4108-8c16-9e1e826bf871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:15.996933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9b6cb-c497-4108-8c16-9e1e826bf871 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:16.021895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 85/171] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:16.022827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9b6cb-c497-4108-8c16-9e1e826bf871 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:16.032758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 87/172] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:16.033736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feda19c-38c6-45ae-a8c8-64fa790b2985 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:16.136035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:16.138275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feda19c-38c6-45ae-a8c8-64fa790b2985 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:16.164809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:16.187815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753aec3-087c-4daf-bb8b-85aa9c243ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:16.188420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753aec3-087c-4daf-bb8b-85aa9c243ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:16.188702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753aec3-087c-4daf-bb8b-85aa9c243ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:16.188971 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753aec3-087c-4daf-bb8b-85aa9c243ca7 None admin] PATH_INFO: `/v3/users/32192a9fb66c412e97d89044a31c5688` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:16.190236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753aec3-087c-4daf-bb8b-85aa9c243ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=32192a9fb66c412e97d89044a31c5688)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:16.191750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753aec3-087c-4daf-bb8b-85aa9c243ca7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:16.193600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 88/173] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:16 2022] GET /identity/v3/users/32192a9fb66c412e97d89044a31c5688 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:16.194191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753aec3-087c-4daf-bb8b-85aa9c243ca7 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:16.202055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9b6cb-c497-4108-8c16-9e1e826bf871 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:16.250184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d98e69-f168-4cbf-8b2e-e5a193ba4853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:16.251229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d98e69-f168-4cbf-8b2e-e5a193ba4853 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:16.251715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d98e69-f168-4cbf-8b2e-e5a193ba4853 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:16.252172 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d98e69-f168-4cbf-8b2e-e5a193ba4853 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:16.253700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d98e69-f168-4cbf-8b2e-e5a193ba4853 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:16.255362 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d98e69-f168-4cbf-8b2e-e5a193ba4853 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:16.264678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 86/174] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:16 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:16.266084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d98e69-f168-4cbf-8b2e-e5a193ba4853 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:16.337391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753aec3-087c-4daf-bb8b-85aa9c243ca7 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:16.410266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09634e07-1ede-4248-a17e-f306ef8c15b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:16.410926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09634e07-1ede-4248-a17e-f306ef8c15b2 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:16.411207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09634e07-1ede-4248-a17e-f306ef8c15b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:16.411497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09634e07-1ede-4248-a17e-f306ef8c15b2 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:16.413640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09634e07-1ede-4248-a17e-f306ef8c15b2 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:16.415191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09634e07-1ede-4248-a17e-f306ef8c15b2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:16.419776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 89/175] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:16 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:16.420597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09634e07-1ede-4248-a17e-f306ef8c15b2 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:16.478982 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d98e69-f168-4cbf-8b2e-e5a193ba4853 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:16.508922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:16.536344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc24cb-6aea-4798-aca9-3d461b2e81af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:16.536633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:16.537313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc24cb-6aea-4798-aca9-3d461b2e81af None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:16.538030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc24cb-6aea-4798-aca9-3d461b2e81af None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:16.538309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc24cb-6aea-4798-aca9-3d461b2e81af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:16.633677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09634e07-1ede-4248-a17e-f306ef8c15b2 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:16.650045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc24cb-6aea-4798-aca9-3d461b2e81af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:16.651546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc24cb-6aea-4798-aca9-3d461b2e81af None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:16.698151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32117642-1350-4766-9551-4ff11307339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:16.698840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32117642-1350-4766-9551-4ff11307339c None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:16.699117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32117642-1350-4766-9551-4ff11307339c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:16.699386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32117642-1350-4766-9551-4ff11307339c None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1/users/375b4c53a8e94a629964fa9a692859eb/roles/cc9d3730bd5c4edcb1897fc5de4e031d` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:16.702101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32117642-1350-4766-9551-4ff11307339c None admin] RBAC: Authorizing `identity:create_grant(project_id=183e3860f51e45b8828e2c44923ca1f1, user_id=375b4c53a8e94a629964fa9a692859eb, role_id=cc9d3730bd5c4edcb1897fc5de4e031d)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:16.703568 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32117642-1350-4766-9551-4ff11307339c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:16.726431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 90/176] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:50:16 2022] PUT /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1/users/375b4c53a8e94a629964fa9a692859eb/roles/cc9d3730bd5c4edcb1897fc5de4e031d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:16.727318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32117642-1350-4766-9551-4ff11307339c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:16.847016 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 87/177] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:16.852861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc24cb-6aea-4798-aca9-3d461b2e81af None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:16.884525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:16.939270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32117642-1350-4766-9551-4ff11307339c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:16.964714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:17.014920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1336490-bed3-4506-a6e0-f1d28c89bc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:17.015561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336490-bed3-4506-a6e0-f1d28c89bc79 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.016000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336490-bed3-4506-a6e0-f1d28c89bc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.016516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1336490-bed3-4506-a6e0-f1d28c89bc79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.017391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1336490-bed3-4506-a6e0-f1d28c89bc79 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=183e3860f51e45b8828e2c44923ca1f1, user.id=85a5af1109c54a2a80385f00893e3137) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:17.017936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1336490-bed3-4506-a6e0-f1d28c89bc79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:17.019596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1336490-bed3-4506-a6e0-f1d28c89bc79 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:17.026512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 91/178] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:16 2022] GET /identity/v3/role_assignments?user.id=85a5af1109c54a2a80385f00893e3137&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:17.032836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760a156-feb5-4f10-901f-c269fd16dfdc None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.033875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760a156-feb5-4f10-901f-c269fd16dfdc None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.034356 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760a156-feb5-4f10-901f-c269fd16dfdc None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.035992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 92/179] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:17.039354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc24cb-6aea-4798-aca9-3d461b2e81af None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:17.046008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334574c-7314-45ce-ae3e-6dedba493fdb None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.046437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334574c-7314-45ce-ae3e-6dedba493fdb None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.046831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334574c-7314-45ce-ae3e-6dedba493fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.089479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-0334574c-7314-45ce-ae3e-6dedba493fdb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:17.090883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0334574c-7314-45ce-ae3e-6dedba493fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:17.092665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0c31f-4f0e-432e-9700-967b322ad7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:17.093877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0c31f-4f0e-432e-9700-967b322ad7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.094323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0c31f-4f0e-432e-9700-967b322ad7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.094810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f0c31f-4f0e-432e-9700-967b322ad7c8 None admin] PATH_INFO: `/v3/projects/1488de510b514c1a937398d753ff15be` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.097521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0c31f-4f0e-432e-9700-967b322ad7c8 None admin] RBAC: Authorizing `identity:get_project(project_id=1488de510b514c1a937398d753ff15be)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:17.099223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f0c31f-4f0e-432e-9700-967b322ad7c8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:17.101882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 88/180] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:16 2022] GET /identity/v3/projects/1488de510b514c1a937398d753ff15be => generated 312 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:17.110365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0c31f-4f0e-432e-9700-967b322ad7c8 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:17.224542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 93/181] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:17.265000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:17.276497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334574c-7314-45ce-ae3e-6dedba493fdb None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:17.304461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f0c31f-4f0e-432e-9700-967b322ad7c8 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:17.377980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e005a-5f2a-43c5-9cb4-23eca399c558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:17.378552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e005a-5f2a-43c5-9cb4-23eca399c558 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.378834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e005a-5f2a-43c5-9cb4-23eca399c558 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.379110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e005a-5f2a-43c5-9cb4-23eca399c558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.379925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e005a-5f2a-43c5-9cb4-23eca399c558 None admin] RBAC: Adding query filter params (role.id=4f5140de9df4440a8fb63d93249d14d1, scope.project.id=1488de510b514c1a937398d753ff15be, user.id=32192a9fb66c412e97d89044a31c5688) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:17.380232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e005a-5f2a-43c5-9cb4-23eca399c558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:17.381766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e005a-5f2a-43c5-9cb4-23eca399c558 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:17.399710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 89/182] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:17 2022] GET /identity/v3/role_assignments?user.id=32192a9fb66c412e97d89044a31c5688&scope.project.id=1488de510b514c1a937398d753ff15be&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:17.422373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e5a55-e3a1-4454-aeb7-117b9705123d None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.422957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e5a55-e3a1-4454-aeb7-117b9705123d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.423354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e5a55-e3a1-4454-aeb7-117b9705123d None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.424993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 90/183] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:17.431670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8334d-c5c0-4c33-9d70-169c42fb3b39 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.431670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8334d-c5c0-4c33-9d70-169c42fb3b39 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.431670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8334d-c5c0-4c33-9d70-169c42fb3b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.445820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334574c-7314-45ce-ae3e-6dedba493fdb None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:17.461188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6ae8334d-c5c0-4c33-9d70-169c42fb3b39 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:17.462789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6ae8334d-c5c0-4c33-9d70-169c42fb3b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:17.467797 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:17.509108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d2645f-0f72-483c-a63c-2f8cf98ca95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:17.509856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2645f-0f72-483c-a63c-2f8cf98ca95a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.510386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2645f-0f72-483c-a63c-2f8cf98ca95a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.510666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d2645f-0f72-483c-a63c-2f8cf98ca95a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.525952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d2645f-0f72-483c-a63c-2f8cf98ca95a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-45d2645f-0f72-483c-a63c-2f8cf98ca95a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:17.533312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:17.534480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:17.535359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 94/184] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:50:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:17.536070 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022df096-3e08-4c6c-9086-ebf87433b589 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.536135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022df096-3e08-4c6c-9086-ebf87433b589 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.536441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022df096-3e08-4c6c-9086-ebf87433b589 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.537990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 95/185] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:17.543152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022df096-3e08-4c6c-9086-ebf87433b589 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:17.564901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 91/186] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:17.566810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93984915-0619-4c58-bde2-089a94e8af77 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.567348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93984915-0619-4c58-bde2-089a94e8af77 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.567795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93984915-0619-4c58-bde2-089a94e8af77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.589257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-93984915-0619-4c58-bde2-089a94e8af77 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:17.590920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-93984915-0619-4c58-bde2-089a94e8af77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:17.673297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022df096-3e08-4c6c-9086-ebf87433b589 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:17.685440 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 92/187] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:17.686701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93984915-0619-4c58-bde2-089a94e8af77 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:17.698190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:17.721095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd9125-9a20-4d0f-b872-51c10040aeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:17.722582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd9125-9a20-4d0f-b872-51c10040aeaf None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.722870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd9125-9a20-4d0f-b872-51c10040aeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.723147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd9125-9a20-4d0f-b872-51c10040aeaf None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.724019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd9125-9a20-4d0f-b872-51c10040aeaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:17.724353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd9125-9a20-4d0f-b872-51c10040aeaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:17.726284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd9125-9a20-4d0f-b872-51c10040aeaf None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:17.736415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 96/188] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:17.737497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd9125-9a20-4d0f-b872-51c10040aeaf None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:17.776141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:17.822944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93984915-0619-4c58-bde2-089a94e8af77 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:17.861176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd9125-9a20-4d0f-b872-51c10040aeaf None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:17.864486 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2e7e7-5e3c-4a44-bcbb-3bc5d93ab46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:17.865153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2e7e7-5e3c-4a44-bcbb-3bc5d93ab46a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.865419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2e7e7-5e3c-4a44-bcbb-3bc5d93ab46a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.865687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2e7e7-5e3c-4a44-bcbb-3bc5d93ab46a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.876958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2e7e7-5e3c-4a44-bcbb-3bc5d93ab46a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-00a2e7e7-5e3c-4a44-bcbb-3bc5d93ab46a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:17.881601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:17.882500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 93/189] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:17.888932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2e7e7-5e3c-4a44-bcbb-3bc5d93ab46a None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:17.896982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763ed96-055f-4da8-af6f-1b35ff0b5df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:17.897662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763ed96-055f-4da8-af6f-1b35ff0b5df2 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:17.898027 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763ed96-055f-4da8-af6f-1b35ff0b5df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:17.898304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763ed96-055f-4da8-af6f-1b35ff0b5df2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:17.899645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763ed96-055f-4da8-af6f-1b35ff0b5df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:17.901345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763ed96-055f-4da8-af6f-1b35ff0b5df2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:17.903341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 97/190] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:17.903918 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763ed96-055f-4da8-af6f-1b35ff0b5df2 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:17.963921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:18.016501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2e7e7-5e3c-4a44-bcbb-3bc5d93ab46a None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:18.021347 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:18.026933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763ed96-055f-4da8-af6f-1b35ff0b5df2 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:18.053437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1d3c4-fb91-4de1-a4ba-c520679bb1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:18.054085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1d3c4-fb91-4de1-a4ba-c520679bb1eb None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:18.054363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1d3c4-fb91-4de1-a4ba-c520679bb1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:18.054638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1d3c4-fb91-4de1-a4ba-c520679bb1eb None admin] PATH_INFO: `/v3/users/d03e575f9de24755b9620b237ad12ff8` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:18.064257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9d67a-6b20-471b-83bb-634a146164b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:18.064906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9d67a-6b20-471b-83bb-634a146164b3 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:18.065177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9d67a-6b20-471b-83bb-634a146164b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:18.065442 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9d67a-6b20-471b-83bb-634a146164b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:18.066189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9d67a-6b20-471b-83bb-634a146164b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:18.066491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9d67a-6b20-471b-83bb-634a146164b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:18.067874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d9d67a-6b20-471b-83bb-634a146164b3 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:18.070893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1d3c4-fb91-4de1-a4ba-c520679bb1eb None admin] RBAC: Authorizing `identity:get_user(user_id=d03e575f9de24755b9620b237ad12ff8)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:18.072610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1d3c4-fb91-4de1-a4ba-c520679bb1eb None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:18.074522 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 94/191] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:17 2022] GET /identity/v3/users/d03e575f9de24755b9620b237ad12ff8 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:18.075281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1d3c4-fb91-4de1-a4ba-c520679bb1eb None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:18.081141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 98/192] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:18.081141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9d67a-6b20-471b-83bb-634a146164b3 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:18.191511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1d3c4-fb91-4de1-a4ba-c520679bb1eb None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:18.203656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d9d67a-6b20-471b-83bb-634a146164b3 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:18.219600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:18.233299 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9becb19-b4b6-43e8-b183-71c66e75cd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:18.233982 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9becb19-b4b6-43e8-b183-71c66e75cd1f None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:18.234265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9becb19-b4b6-43e8-b183-71c66e75cd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:18.234537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9becb19-b4b6-43e8-b183-71c66e75cd1f None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:18.235238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9becb19-b4b6-43e8-b183-71c66e75cd1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:18.236740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9becb19-b4b6-43e8-b183-71c66e75cd1f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:18.241700 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153118b-10ad-4429-8635-3bf4e267725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:18.242336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153118b-10ad-4429-8635-3bf4e267725b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:18.242616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153118b-10ad-4429-8635-3bf4e267725b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:18.242935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153118b-10ad-4429-8635-3bf4e267725b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:18.261258 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 95/193] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:50:18 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:18.262489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9becb19-b4b6-43e8-b183-71c66e75cd1f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:18.290674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:18.341307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153118b-10ad-4429-8635-3bf4e267725b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:18.342900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153118b-10ad-4429-8635-3bf4e267725b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:18.398251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9becb19-b4b6-43e8-b183-71c66e75cd1f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:18.440022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1b61e-b354-48ec-a49d-14b0c1aed073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:18.441388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1b61e-b354-48ec-a49d-14b0c1aed073 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:18.441388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1b61e-b354-48ec-a49d-14b0c1aed073 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:18.441388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1b61e-b354-48ec-a49d-14b0c1aed073 None admin] PATH_INFO: `/v3/users/d03e575f9de24755b9620b237ad12ff8` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:18.442683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1b61e-b354-48ec-a49d-14b0c1aed073 None admin] RBAC: Authorizing `identity:get_user(user_id=d03e575f9de24755b9620b237ad12ff8)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:18.444697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1b61e-b354-48ec-a49d-14b0c1aed073 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:18.446581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 96/194] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:18 2022] GET /identity/v3/users/d03e575f9de24755b9620b237ad12ff8 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:18.453369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1b61e-b354-48ec-a49d-14b0c1aed073 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:18.468186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 99/195] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:18.473711 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153118b-10ad-4429-8635-3bf4e267725b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:18.520230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:18.550763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:18.585156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1b61e-b354-48ec-a49d-14b0c1aed073 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:18.601873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153118b-10ad-4429-8635-3bf4e267725b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:18.631746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05a233-1b9b-4459-92d1-26cfa45a6fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:18.632438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05a233-1b9b-4459-92d1-26cfa45a6fee None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:18.632722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05a233-1b9b-4459-92d1-26cfa45a6fee None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:18.632997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05a233-1b9b-4459-92d1-26cfa45a6fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:18.639994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f1e65-175b-44a6-9bc8-a1dcbcc73e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:18.640680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f1e65-175b-44a6-9bc8-a1dcbcc73e65 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:18.640959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f1e65-175b-44a6-9bc8-a1dcbcc73e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:18.641231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f1e65-175b-44a6-9bc8-a1dcbcc73e65 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:18.644351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f1e65-175b-44a6-9bc8-a1dcbcc73e65 None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:18.645834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f1e65-175b-44a6-9bc8-a1dcbcc73e65 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:18.647875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 100/196] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:18 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:18.657581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f1e65-175b-44a6-9bc8-a1dcbcc73e65 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:18.710122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05a233-1b9b-4459-92d1-26cfa45a6fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:18.711736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05a233-1b9b-4459-92d1-26cfa45a6fee None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:18.772983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:18.805830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f1e65-175b-44a6-9bc8-a1dcbcc73e65 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:18.842363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 97/197] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:18.848557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05a233-1b9b-4459-92d1-26cfa45a6fee None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:18.859128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:18.872358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee549aed-f4b2-4a19-82e5-1fc6d9600f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:18.874430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee549aed-f4b2-4a19-82e5-1fc6d9600f15 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:18.875189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee549aed-f4b2-4a19-82e5-1fc6d9600f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:18.875720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee549aed-f4b2-4a19-82e5-1fc6d9600f15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:18.877398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee549aed-f4b2-4a19-82e5-1fc6d9600f15 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=876fcd5989ee455a89866281d214740e, user.id=d03e575f9de24755b9620b237ad12ff8) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:18.878040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee549aed-f4b2-4a19-82e5-1fc6d9600f15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:18.880433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee549aed-f4b2-4a19-82e5-1fc6d9600f15 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:18.887127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 101/198] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:18 2022] GET /identity/v3/role_assignments?user.id=d03e575f9de24755b9620b237ad12ff8&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:19.009002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05a233-1b9b-4459-92d1-26cfa45a6fee None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:19.062377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de39b4d6-3a1f-48da-b3e9-abf68ec02fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:19.063035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39b4d6-3a1f-48da-b3e9-abf68ec02fcd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:19.063316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39b4d6-3a1f-48da-b3e9-abf68ec02fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:19.063667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de39b4d6-3a1f-48da-b3e9-abf68ec02fcd None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:19.069305 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39b4d6-3a1f-48da-b3e9-abf68ec02fcd None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:19.070871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de39b4d6-3a1f-48da-b3e9-abf68ec02fcd None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:19.072962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 98/199] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:18 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:19.079365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee549aed-f4b2-4a19-82e5-1fc6d9600f15 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:19.216782 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee549aed-f4b2-4a19-82e5-1fc6d9600f15 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:19.256371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6492f-db4c-4117-8dbc-ec85bc5763a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:19.257022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6492f-db4c-4117-8dbc-ec85bc5763a6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:19.257297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6492f-db4c-4117-8dbc-ec85bc5763a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:19.257569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6492f-db4c-4117-8dbc-ec85bc5763a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:19.258449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6492f-db4c-4117-8dbc-ec85bc5763a6 None admin] RBAC: Adding query filter params (role.id=8e13cd96bb124d52b5c8883fe960424f, scope.project.id=876fcd5989ee455a89866281d214740e, user.id=d03e575f9de24755b9620b237ad12ff8) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:19.258757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6492f-db4c-4117-8dbc-ec85bc5763a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:19.260268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6492f-db4c-4117-8dbc-ec85bc5763a6 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:19.267580 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 102/200] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:19 2022] GET /identity/v3/role_assignments?user.id=d03e575f9de24755b9620b237ad12ff8&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=8e13cd96bb124d52b5c8883fe960424f => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:19.559098 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b184b1-b7e3-4910-a21c-be4ae03d7550 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:19.559628 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b184b1-b7e3-4910-a21c-be4ae03d7550 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:19.559711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b184b1-b7e3-4910-a21c-be4ae03d7550 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:19.561457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 99/201] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:19.567352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53436b92-ff03-40b6-94a5-90481162632e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:19.567641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53436b92-ff03-40b6-94a5-90481162632e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:19.567918 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53436b92-ff03-40b6-94a5-90481162632e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:19.597538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-53436b92-ff03-40b6-94a5-90481162632e None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:19.598786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-53436b92-ff03-40b6-94a5-90481162632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:19.671039 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:19.693091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 103/202] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:19.729648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b184b1-b7e3-4910-a21c-be4ae03d7550 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:19.757194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:19.856432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b184b1-b7e3-4910-a21c-be4ae03d7550 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:19.893848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecd7ec-d65f-41fa-afad-b2b735d756de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:19.894523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecd7ec-d65f-41fa-afad-b2b735d756de None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:19.894806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecd7ec-d65f-41fa-afad-b2b735d756de None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:19.895081 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aecd7ec-d65f-41fa-afad-b2b735d756de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:19.906828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aecd7ec-d65f-41fa-afad-b2b735d756de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-0aecd7ec-d65f-41fa-afad-b2b735d756de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:19.911955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:19.912865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:19.913876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 100/203] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:19.923258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53436b92-ff03-40b6-94a5-90481162632e None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:20.071417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53436b92-ff03-40b6-94a5-90481162632e None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:20.117686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3d64b-a549-4f62-95b6-fbf04db56cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:20.118298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3d64b-a549-4f62-95b6-fbf04db56cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:20.118537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3d64b-a549-4f62-95b6-fbf04db56cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:20.118784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3d64b-a549-4f62-95b6-fbf04db56cc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:20.119539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3d64b-a549-4f62-95b6-fbf04db56cc9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:20.119844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3d64b-a549-4f62-95b6-fbf04db56cc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:20.121275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3d64b-a549-4f62-95b6-fbf04db56cc9 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:20.130778 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 104/204] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:20.136490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecd7ec-d65f-41fa-afad-b2b735d756de None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:20.274301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:20.306502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aecd7ec-d65f-41fa-afad-b2b735d756de None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:20.353785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6149f-b269-4b12-aa4f-5471bddb2212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:20.354474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6149f-b269-4b12-aa4f-5471bddb2212 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:20.354755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6149f-b269-4b12-aa4f-5471bddb2212 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:20.355027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f6149f-b269-4b12-aa4f-5471bddb2212 None admin] PATH_INFO: `/v3/users/375b4c53a8e94a629964fa9a692859eb` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:20.361341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6149f-b269-4b12-aa4f-5471bddb2212 None admin] RBAC: Authorizing `identity:get_user(user_id=375b4c53a8e94a629964fa9a692859eb)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:20.363282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f6149f-b269-4b12-aa4f-5471bddb2212 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:20.365418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 101/205] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:20 2022] GET /identity/v3/users/375b4c53a8e94a629964fa9a692859eb => generated 289 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:20.371051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3d64b-a549-4f62-95b6-fbf04db56cc9 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:20.447647 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:20.523084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3d64b-a549-4f62-95b6-fbf04db56cc9 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:20.559827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a0143-c24f-4f44-a3aa-67101a38319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:20.560498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a0143-c24f-4f44-a3aa-67101a38319a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:20.560776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a0143-c24f-4f44-a3aa-67101a38319a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:20.561047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a0143-c24f-4f44-a3aa-67101a38319a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:20.652658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a0143-c24f-4f44-a3aa-67101a38319a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:20.657219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a0143-c24f-4f44-a3aa-67101a38319a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:20.719240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:20.797876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 105/206] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:20.803226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6149f-b269-4b12-aa4f-5471bddb2212 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:20.901569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a38a7f-022c-47e3-8423-44519c380dec None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:20.901888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a38a7f-022c-47e3-8423-44519c380dec None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:20.902124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a38a7f-022c-47e3-8423-44519c380dec None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:20.903581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 106/207] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:20.909286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb979b-dcb5-4d75-8afc-ccfb173f3183 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:20.909568 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb979b-dcb5-4d75-8afc-ccfb173f3183 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:20.909839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb979b-dcb5-4d75-8afc-ccfb173f3183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:20.937661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-28cb979b-dcb5-4d75-8afc-ccfb173f3183 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:20.939368 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-28cb979b-dcb5-4d75-8afc-ccfb173f3183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:20.969928 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f6149f-b269-4b12-aa4f-5471bddb2212 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:21.028325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a2dea-bd99-4a03-9dba-519c25fe0858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:21.029164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a2dea-bd99-4a03-9dba-519c25fe0858 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.029451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a2dea-bd99-4a03-9dba-519c25fe0858 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.029724 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a2dea-bd99-4a03-9dba-519c25fe0858 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.034445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a2dea-bd99-4a03-9dba-519c25fe0858 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:21.034445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a2dea-bd99-4a03-9dba-519c25fe0858 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:21.034445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 102/208] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:20 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:21.040508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a2dea-bd99-4a03-9dba-519c25fe0858 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:21.057497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:21.129729 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 107/209] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:21.153412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a2dea-bd99-4a03-9dba-519c25fe0858 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:21.165509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb979b-dcb5-4d75-8afc-ccfb173f3183 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:21.193321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9792a-1e36-43b7-b7a9-163ad756b777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:21.194701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9792a-1e36-43b7-b7a9-163ad756b777 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.195184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9792a-1e36-43b7-b7a9-163ad756b777 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.195723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9792a-1e36-43b7-b7a9-163ad756b777 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.196825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9792a-1e36-43b7-b7a9-163ad756b777 None admin] RBAC: Adding query filter params (role.id=cc9d3730bd5c4edcb1897fc5de4e031d, scope.project.id=183e3860f51e45b8828e2c44923ca1f1, user.id=375b4c53a8e94a629964fa9a692859eb) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:21.197504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9792a-1e36-43b7-b7a9-163ad756b777 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:21.199202 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9792a-1e36-43b7-b7a9-163ad756b777 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:21.206286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 103/210] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:21 2022] GET /identity/v3/role_assignments?user.id=375b4c53a8e94a629964fa9a692859eb&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=cc9d3730bd5c4edcb1897fc5de4e031d => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:21.276745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:21.317206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb979b-dcb5-4d75-8afc-ccfb173f3183 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:21.378502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a1375-1ca1-427e-b09f-d52f9a9bd243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:21.379187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a1375-1ca1-427e-b09f-d52f9a9bd243 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.379465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a1375-1ca1-427e-b09f-d52f9a9bd243 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.379735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a1375-1ca1-427e-b09f-d52f9a9bd243 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.396449 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7183aff-d633-482d-9baa-4e614ee2f329 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.396819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7183aff-d633-482d-9baa-4e614ee2f329 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.397124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7183aff-d633-482d-9baa-4e614ee2f329 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.398795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 104/211] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:21.401015 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a1375-1ca1-427e-b09f-d52f9a9bd243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-d40a1375-1ca1-427e-b09f-d52f9a9bd243 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:21.406113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:21.406976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:21.407950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 108/212] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:21.411661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef3e7e-ca8a-4adb-966e-f6a8c21ae220 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.411975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef3e7e-ca8a-4adb-966e-f6a8c21ae220 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.412409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef3e7e-ca8a-4adb-966e-f6a8c21ae220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.413732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a1375-1ca1-427e-b09f-d52f9a9bd243 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:21.441023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-71ef3e7e-ca8a-4adb-966e-f6a8c21ae220 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:21.442145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-71ef3e7e-ca8a-4adb-966e-f6a8c21ae220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:21.529854 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a1375-1ca1-427e-b09f-d52f9a9bd243 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:21.538359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 105/213] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:21.540408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef2531-712b-40a6-9a1e-11711c133b10 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.540702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef2531-712b-40a6-9a1e-11711c133b10 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.540981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef2531-712b-40a6-9a1e-11711c133b10 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.542698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 106/214] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:21.548443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1814-04f5-459a-83d5-4d358116ea24 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.548443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1814-04f5-459a-83d5-4d358116ea24 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.548825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbd1814-04f5-459a-83d5-4d358116ea24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.561574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:21.568757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-ddbd1814-04f5-459a-83d5-4d358116ea24 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:21.569970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ddbd1814-04f5-459a-83d5-4d358116ea24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:21.573385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dd756-c70d-4818-80c8-d558d021286e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:21.574030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dd756-c70d-4818-80c8-d558d021286e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.574310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dd756-c70d-4818-80c8-d558d021286e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.574584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752dd756-c70d-4818-80c8-d558d021286e None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.575313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752dd756-c70d-4818-80c8-d558d021286e None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:21.575620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752dd756-c70d-4818-80c8-d558d021286e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:21.577149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752dd756-c70d-4818-80c8-d558d021286e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:21.583554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:21.592624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 109/215] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:21.593493 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dd756-c70d-4818-80c8-d558d021286e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:21.645145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 107/216] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:21.645819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd1814-04f5-459a-83d5-4d358116ea24 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:21.720773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752dd756-c70d-4818-80c8-d558d021286e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:21.762193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b166d-f08b-4d47-83ec-2d6aab9d70ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:21.762960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b166d-f08b-4d47-83ec-2d6aab9d70ca None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.763244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b166d-f08b-4d47-83ec-2d6aab9d70ca None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.763519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b166d-f08b-4d47-83ec-2d6aab9d70ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.774964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbd1814-04f5-459a-83d5-4d358116ea24 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:21.775852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b166d-f08b-4d47-83ec-2d6aab9d70ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-911b166d-f08b-4d47-83ec-2d6aab9d70ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:21.782619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:21.783672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:21.784630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 110/217] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:50:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:21.786749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b166d-f08b-4d47-83ec-2d6aab9d70ca None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:21.817684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e47e4f-7cf7-4c17-b54e-29431f0e6e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:21.818448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e47e4f-7cf7-4c17-b54e-29431f0e6e09 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.818626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e47e4f-7cf7-4c17-b54e-29431f0e6e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.818876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e47e4f-7cf7-4c17-b54e-29431f0e6e09 None admin] PATH_INFO: `/v3/users/009c40702c464facadbc0960b59e505f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.822267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:21.836448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e47e4f-7cf7-4c17-b54e-29431f0e6e09 None admin] RBAC: Authorizing `identity:get_user(user_id=009c40702c464facadbc0960b59e505f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:21.837944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e47e4f-7cf7-4c17-b54e-29431f0e6e09 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:21.839673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 108/218] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:21 2022] GET /identity/v3/users/009c40702c464facadbc0960b59e505f => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:21.840913 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0a87a-e128-4c52-8a56-71b1afcca7dc None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.841195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0a87a-e128-4c52-8a56-71b1afcca7dc None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.841465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0a87a-e128-4c52-8a56-71b1afcca7dc None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.842709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 109/219] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:21.847729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0a87a-e128-4c52-8a56-71b1afcca7dc None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:21.861404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:21.905762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b166d-f08b-4d47-83ec-2d6aab9d70ca None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:21.939747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4147a7-684c-4e0f-bdfc-b2458ab46c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:21.940423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4147a7-684c-4e0f-bdfc-b2458ab46c8e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.940697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4147a7-684c-4e0f-bdfc-b2458ab46c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.940971 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4147a7-684c-4e0f-bdfc-b2458ab46c8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.942139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4147a7-684c-4e0f-bdfc-b2458ab46c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:21.943622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4147a7-684c-4e0f-bdfc-b2458ab46c8e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:21.945265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 111/220] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:21.946134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4147a7-684c-4e0f-bdfc-b2458ab46c8e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:21.958451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0a87a-e128-4c52-8a56-71b1afcca7dc None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:21.994065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb25bf-a4d3-403a-a08f-4714036a396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:21.994950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb25bf-a4d3-403a-a08f-4714036a396f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:21.995230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb25bf-a4d3-403a-a08f-4714036a396f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:21.995501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb25bf-a4d3-403a-a08f-4714036a396f None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:21.996232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb25bf-a4d3-403a-a08f-4714036a396f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:21.996568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb25bf-a4d3-403a-a08f-4714036a396f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:21.997976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb25bf-a4d3-403a-a08f-4714036a396f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:22.006880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 110/221] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:22.008136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00a092-a5e1-47ef-9922-8c3a6c58e8a6 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:22.008467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00a092-a5e1-47ef-9922-8c3a6c58e8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:22.008744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00a092-a5e1-47ef-9922-8c3a6c58e8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:22.026938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-dc00a092-a5e1-47ef-9922-8c3a6c58e8a6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:22.028146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-dc00a092-a5e1-47ef-9922-8c3a6c58e8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:22.055397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4147a7-684c-4e0f-bdfc-b2458ab46c8e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:22.079439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:22.083901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:22.093588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41984021-2c37-4faf-b08a-e98faba62109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:22.094220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41984021-2c37-4faf-b08a-e98faba62109 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:22.094501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41984021-2c37-4faf-b08a-e98faba62109 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:22.094776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41984021-2c37-4faf-b08a-e98faba62109 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:22.099694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 111/222] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:22.100578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00a092-a5e1-47ef-9922-8c3a6c58e8a6 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:22.179514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41984021-2c37-4faf-b08a-e98faba62109 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:22.183355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41984021-2c37-4faf-b08a-e98faba62109 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:22.209291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00a092-a5e1-47ef-9922-8c3a6c58e8a6 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:22.245507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6787a-99ff-4f8f-a824-6e4781dc861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:22.246052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6787a-99ff-4f8f-a824-6e4781dc861e None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:22.246334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6787a-99ff-4f8f-a824-6e4781dc861e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:22.246608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6787a-99ff-4f8f-a824-6e4781dc861e None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:22.247296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6787a-99ff-4f8f-a824-6e4781dc861e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:22.248800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6787a-99ff-4f8f-a824-6e4781dc861e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:22.284760 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 112/223] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:50:22 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:22.284760 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6787a-99ff-4f8f-a824-6e4781dc861e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:22.301411 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 112/224] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:22.302409 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41984021-2c37-4faf-b08a-e98faba62109 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:22.324598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:22.338259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:22.398455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6787a-99ff-4f8f-a824-6e4781dc861e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:22.428110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41984021-2c37-4faf-b08a-e98faba62109 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:22.446509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5caff-249c-4f9a-aa17-17f71967bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:22.447175 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5caff-249c-4f9a-aa17-17f71967bd7d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:22.447452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5caff-249c-4f9a-aa17-17f71967bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:22.447723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5caff-249c-4f9a-aa17-17f71967bd7d None admin] PATH_INFO: `/v3/users/d03e575f9de24755b9620b237ad12ff8` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:22.448991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5caff-249c-4f9a-aa17-17f71967bd7d None admin] RBAC: Authorizing `identity:get_user(user_id=d03e575f9de24755b9620b237ad12ff8)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:22.450469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5caff-249c-4f9a-aa17-17f71967bd7d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:22.452213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 113/225] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:22 2022] GET /identity/v3/users/d03e575f9de24755b9620b237ad12ff8 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:22.453155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5caff-249c-4f9a-aa17-17f71967bd7d None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:22.484396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b98b3b-5853-4789-88e3-8a456a4fc4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:22.485150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b98b3b-5853-4789-88e3-8a456a4fc4ba None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:22.485425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b98b3b-5853-4789-88e3-8a456a4fc4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:22.485839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b98b3b-5853-4789-88e3-8a456a4fc4ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:22.499577 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b98b3b-5853-4789-88e3-8a456a4fc4ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-e7b98b3b-5853-4789-88e3-8a456a4fc4ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:22.504726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:22.505620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:22.506391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 113/226] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:22.507040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b98b3b-5853-4789-88e3-8a456a4fc4ba None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:22.603683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5caff-249c-4f9a-aa17-17f71967bd7d None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:22.619012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:22.626721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:22.646150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b98b3b-5853-4789-88e3-8a456a4fc4ba None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:22.653465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e58a82-b7f4-477e-983f-0d117f645b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:22.654292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e58a82-b7f4-477e-983f-0d117f645b43 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:22.654576 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e58a82-b7f4-477e-983f-0d117f645b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:22.654862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e58a82-b7f4-477e-983f-0d117f645b43 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:22.656734 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e58a82-b7f4-477e-983f-0d117f645b43 None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:22.658631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e58a82-b7f4-477e-983f-0d117f645b43 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:22.661397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 114/227] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:22 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:22.662168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e58a82-b7f4-477e-983f-0d117f645b43 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:22.698769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f62b9-bb78-440e-8b97-0c642c904955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:22.699873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f62b9-bb78-440e-8b97-0c642c904955 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:22.700425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f62b9-bb78-440e-8b97-0c642c904955 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:22.700911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f62b9-bb78-440e-8b97-0c642c904955 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:22.702964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f62b9-bb78-440e-8b97-0c642c904955 None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:22.705769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f62b9-bb78-440e-8b97-0c642c904955 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:22.709247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 114/228] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:22 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:22.710555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f62b9-bb78-440e-8b97-0c642c904955 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:22.829090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e58a82-b7f4-477e-983f-0d117f645b43 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:22.857707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f62b9-bb78-440e-8b97-0c642c904955 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:22.878272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabed1c8-18fe-42a7-a8d6-82b4dc66e5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:22.879309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabed1c8-18fe-42a7-a8d6-82b4dc66e5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:22.879775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabed1c8-18fe-42a7-a8d6-82b4dc66e5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:22.880135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabed1c8-18fe-42a7-a8d6-82b4dc66e5b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:22.880927 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabed1c8-18fe-42a7-a8d6-82b4dc66e5b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:22.881329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabed1c8-18fe-42a7-a8d6-82b4dc66e5b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:22.882752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabed1c8-18fe-42a7-a8d6-82b4dc66e5b7 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:22.893307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 115/229] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:22.894101 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabed1c8-18fe-42a7-a8d6-82b4dc66e5b7 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:22.908434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f32e3-c1e5-4d24-b8dc-ba700eaa5520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:22.909352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f32e3-c1e5-4d24-b8dc-ba700eaa5520 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:22.909659 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f32e3-c1e5-4d24-b8dc-ba700eaa5520 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:22.910372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508f32e3-c1e5-4d24-b8dc-ba700eaa5520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:22.911995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f32e3-c1e5-4d24-b8dc-ba700eaa5520 None admin] RBAC: Adding query filter params (role.id=4f5140de9df4440a8fb63d93249d14d1, scope.project.id=876fcd5989ee455a89866281d214740e, user.id=009c40702c464facadbc0960b59e505f) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:22.912571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f32e3-c1e5-4d24-b8dc-ba700eaa5520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:22.915194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508f32e3-c1e5-4d24-b8dc-ba700eaa5520 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:22.923352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 115/230] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:22 2022] GET /identity/v3/role_assignments?user.id=009c40702c464facadbc0960b59e505f&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:22.924426 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f32e3-c1e5-4d24-b8dc-ba700eaa5520 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:22.963187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:22.967541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:23.075042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508f32e3-c1e5-4d24-b8dc-ba700eaa5520 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:23.106218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabed1c8-18fe-42a7-a8d6-82b4dc66e5b7 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:23.130991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703041a6-0f9f-4cb7-847c-bf4930f7dfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:23.131698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703041a6-0f9f-4cb7-847c-bf4930f7dfc5 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:23.131981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703041a6-0f9f-4cb7-847c-bf4930f7dfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:23.132319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703041a6-0f9f-4cb7-847c-bf4930f7dfc5 None admin] PATH_INFO: `/v3/users/d03e575f9de24755b9620b237ad12ff8` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:23.133729 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703041a6-0f9f-4cb7-847c-bf4930f7dfc5 None admin] RBAC: Authorizing `identity:get_user(user_id=d03e575f9de24755b9620b237ad12ff8)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:23.135200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703041a6-0f9f-4cb7-847c-bf4930f7dfc5 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:23.141593 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 116/231] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:22 2022] GET /identity/v3/users/d03e575f9de24755b9620b237ad12ff8 => generated 283 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:23.142990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703041a6-0f9f-4cb7-847c-bf4930f7dfc5 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:23.177529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bdfa4-3e39-40a9-b7ec-141425cfea97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:23.178255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bdfa4-3e39-40a9-b7ec-141425cfea97 None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:23.178543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bdfa4-3e39-40a9-b7ec-141425cfea97 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:23.178826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bdfa4-3e39-40a9-b7ec-141425cfea97 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e/users/d03e575f9de24755b9620b237ad12ff8/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:23.185226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bdfa4-3e39-40a9-b7ec-141425cfea97 None admin] RBAC: Authorizing `identity:create_grant(project_id=876fcd5989ee455a89866281d214740e, user_id=d03e575f9de24755b9620b237ad12ff8, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:23.186833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0bdfa4-3e39-40a9-b7ec-141425cfea97 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:23.213852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 116/232] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:50:22 2022] PUT /identity/v3/projects/876fcd5989ee455a89866281d214740e/users/d03e575f9de24755b9620b237ad12ff8/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:23.250808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:23.267271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703041a6-0f9f-4cb7-847c-bf4930f7dfc5 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:23.308928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04efa4-a510-43ab-8ecb-394a97df7a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:23.309606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04efa4-a510-43ab-8ecb-394a97df7a8f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:23.309888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04efa4-a510-43ab-8ecb-394a97df7a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:23.310164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04efa4-a510-43ab-8ecb-394a97df7a8f None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:23.311445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04efa4-a510-43ab-8ecb-394a97df7a8f None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:23.313080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04efa4-a510-43ab-8ecb-394a97df7a8f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:23.320089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 117/233] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:23 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:23.321552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bdfa4-3e39-40a9-b7ec-141425cfea97 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:23.433483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:23.437583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bdfa4-3e39-40a9-b7ec-141425cfea97 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:23.490687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db91068-7a90-4239-8737-575263536844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:23.491389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db91068-7a90-4239-8737-575263536844 None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:23.491671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db91068-7a90-4239-8737-575263536844 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:23.492000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db91068-7a90-4239-8737-575263536844 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e/users/d03e575f9de24755b9620b237ad12ff8/roles/8e13cd96bb124d52b5c8883fe960424f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:23.494586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db91068-7a90-4239-8737-575263536844 None admin] RBAC: Authorizing `identity:create_grant(project_id=876fcd5989ee455a89866281d214740e, user_id=d03e575f9de24755b9620b237ad12ff8, role_id=8e13cd96bb124d52b5c8883fe960424f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:23.496086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db91068-7a90-4239-8737-575263536844 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:23.521964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 117/234] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:50:23 2022] PUT /identity/v3/projects/876fcd5989ee455a89866281d214740e/users/d03e575f9de24755b9620b237ad12ff8/roles/8e13cd96bb124d52b5c8883fe960424f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:24.847059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c383-6d0b-481c-89d2-399d2253cfbe None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:24.847059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c383-6d0b-481c-89d2-399d2253cfbe None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:24.847059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636c383-6d0b-481c-89d2-399d2253cfbe None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:24.847955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 118/235] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:24.857976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74b438-a64c-4ea0-ac87-f68270ab4452 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:24.858875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74b438-a64c-4ea0-ac87-f68270ab4452 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:24.859350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74b438-a64c-4ea0-ac87-f68270ab4452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:24.880220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6b74b438-a64c-4ea0-ac87-f68270ab4452 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:24.881833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6b74b438-a64c-4ea0-ac87-f68270ab4452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:24.904085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b71a88-2c6a-4763-852c-1e3b97a9b8d2 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:24.904443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b71a88-2c6a-4763-852c-1e3b97a9b8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:24.904691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b71a88-2c6a-4763-852c-1e3b97a9b8d2 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:24.906232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 119/236] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:24.911340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47236f8-97cc-411c-aebf-736c1e6172cd None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:24.911626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47236f8-97cc-411c-aebf-736c1e6172cd None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:24.911903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47236f8-97cc-411c-aebf-736c1e6172cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:24.933292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a47236f8-97cc-411c-aebf-736c1e6172cd None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:24.934625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a47236f8-97cc-411c-aebf-736c1e6172cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:24.972940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 118/237] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:25.009134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74b438-a64c-4ea0-ac87-f68270ab4452 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:25.028120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 120/238] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:25.114225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47236f8-97cc-411c-aebf-736c1e6172cd None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:25.148742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:25.171066 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74b438-a64c-4ea0-ac87-f68270ab4452 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:25.243931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3a8d0-61e0-475d-b921-bf26b82c76af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:25.244625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3a8d0-61e0-475d-b921-bf26b82c76af None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:25.244901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3a8d0-61e0-475d-b921-bf26b82c76af None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:25.245171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3a8d0-61e0-475d-b921-bf26b82c76af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:25.249989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3a8d0-61e0-475d-b921-bf26b82c76af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:25.251494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3a8d0-61e0-475d-b921-bf26b82c76af None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:25.253761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 119/239] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:25.260049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3a8d0-61e0-475d-b921-bf26b82c76af None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:25.317998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:25.341474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47236f8-97cc-411c-aebf-736c1e6172cd None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:25.388817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59086540-3a53-47b9-a1ae-f0c59e516e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:25.389006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086540-3a53-47b9-a1ae-f0c59e516e3b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:25.389268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086540-3a53-47b9-a1ae-f0c59e516e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:25.389521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59086540-3a53-47b9-a1ae-f0c59e516e3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:25.404203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59086540-3a53-47b9-a1ae-f0c59e516e3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-59086540-3a53-47b9-a1ae-f0c59e516e3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:25.409730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:25.410639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:25.411641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 121/240] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:25.421352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59086540-3a53-47b9-a1ae-f0c59e516e3b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:25.448956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3a8d0-61e0-475d-b921-bf26b82c76af None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:25.512567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67ad92-f655-4862-9786-fedf029f3199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:25.513604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67ad92-f655-4862-9786-fedf029f3199 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:25.514061 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67ad92-f655-4862-9786-fedf029f3199 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:25.514511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67ad92-f655-4862-9786-fedf029f3199 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:25.515816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67ad92-f655-4862-9786-fedf029f3199 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:25.518598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67ad92-f655-4862-9786-fedf029f3199 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:25.565984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 120/241] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:50:25 2022] POST /identity/v3/users => generated 293 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:25.567887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c1838-166a-41fc-9e1b-ef89291f243e None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:25.568393 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c1838-166a-41fc-9e1b-ef89291f243e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:25.568891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c1838-166a-41fc-9e1b-ef89291f243e None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:25.574978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 121/242] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:25.577398 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdd12d-0d23-4299-95bf-79cc619c5332 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:25.577934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdd12d-0d23-4299-95bf-79cc619c5332 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:25.578438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdd12d-0d23-4299-95bf-79cc619c5332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:25.599086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a7bdd12d-0d23-4299-95bf-79cc619c5332 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:25.600853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a7bdd12d-0d23-4299-95bf-79cc619c5332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:25.615450 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:25.642523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59086540-3a53-47b9-a1ae-f0c59e516e3b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:25.725136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 122/243] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:25.727565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93ceeb-83b3-4caa-8c39-be9698f41e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:25.728485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93ceeb-83b3-4caa-8c39-be9698f41e80 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:25.728967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93ceeb-83b3-4caa-8c39-be9698f41e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:25.729490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93ceeb-83b3-4caa-8c39-be9698f41e80 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:25.730461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93ceeb-83b3-4caa-8c39-be9698f41e80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:25.730968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93ceeb-83b3-4caa-8c39-be9698f41e80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:25.732642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93ceeb-83b3-4caa-8c39-be9698f41e80 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:25.754280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 122/244] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:25.760511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdd12d-0d23-4299-95bf-79cc619c5332 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:25.782476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93ceeb-83b3-4caa-8c39-be9698f41e80 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:25.798767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:25.926229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdd12d-0d23-4299-95bf-79cc619c5332 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:25.940393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93ceeb-83b3-4caa-8c39-be9698f41e80 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:25.963124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d7051-e9af-4280-a7c5-16a486b5445f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:25.963791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d7051-e9af-4280-a7c5-16a486b5445f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:25.964130 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d7051-e9af-4280-a7c5-16a486b5445f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:25.964468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d7051-e9af-4280-a7c5-16a486b5445f None admin] PATH_INFO: `/v3/users/2f3d14ad9e6c4fdda59307e744e32b02` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:25.979426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d7051-e9af-4280-a7c5-16a486b5445f None admin] RBAC: Authorizing `identity:get_user(user_id=2f3d14ad9e6c4fdda59307e744e32b02)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:25.980944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4117b-d5a4-4628-a69f-e271f670d3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:25.981080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d7051-e9af-4280-a7c5-16a486b5445f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:25.981953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4117b-d5a4-4628-a69f-e271f670d3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:25.982433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4117b-d5a4-4628-a69f-e271f670d3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:25.982897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4117b-d5a4-4628-a69f-e271f670d3c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:25.983165 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 123/245] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:25 2022] GET /identity/v3/users/2f3d14ad9e6c4fdda59307e744e32b02 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:25.989017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d7051-e9af-4280-a7c5-16a486b5445f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:25.991970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:25.998186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f4117b-d5a4-4628-a69f-e271f670d3c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-a0f4117b-d5a4-4628-a69f-e271f670d3c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:26.005337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:26.006277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:26.007041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:26.009800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 123/246] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:26.017108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4117b-d5a4-4628-a69f-e271f670d3c7 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:26.099444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:26.106697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d7051-e9af-4280-a7c5-16a486b5445f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:26.138867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4117b-d5a4-4628-a69f-e271f670d3c7 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:26.145704 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1bf9e-4c3d-4b37-a7f6-0efdd9b9e779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:26.146337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1bf9e-4c3d-4b37-a7f6-0efdd9b9e779 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.146619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1bf9e-4c3d-4b37-a7f6-0efdd9b9e779 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.146888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1bf9e-4c3d-4b37-a7f6-0efdd9b9e779 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.179440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1697a8-49d1-4700-8f97-6e33ded36a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:26.180637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1697a8-49d1-4700-8f97-6e33ded36a85 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.181119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1697a8-49d1-4700-8f97-6e33ded36a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.181588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1697a8-49d1-4700-8f97-6e33ded36a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.182624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1697a8-49d1-4700-8f97-6e33ded36a85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:26.183217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1697a8-49d1-4700-8f97-6e33ded36a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:26.185038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1697a8-49d1-4700-8f97-6e33ded36a85 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:26.195341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 124/247] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:26.202744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1697a8-49d1-4700-8f97-6e33ded36a85 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:26.233125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1bf9e-4c3d-4b37-a7f6-0efdd9b9e779 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:26.235424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1bf9e-4c3d-4b37-a7f6-0efdd9b9e779 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:26.240592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:26.346330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1697a8-49d1-4700-8f97-6e33ded36a85 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:26.377602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 124/248] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 11:50:26.379049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7792c-cdb2-40dd-87a1-c80b5f427933 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.379334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7792c-cdb2-40dd-87a1-c80b5f427933 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.379604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7792c-cdb2-40dd-87a1-c80b5f427933 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.386591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:26.387164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 125/249] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:26.388256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7792c-cdb2-40dd-87a1-c80b5f427933 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:26.399251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9b5f4-a0cb-4bc8-9eb6-48b30b826e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:26.400361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9b5f4-a0cb-4bc8-9eb6-48b30b826e4c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.400869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9b5f4-a0cb-4bc8-9eb6-48b30b826e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.401337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9b5f4-a0cb-4bc8-9eb6-48b30b826e4c None admin] PATH_INFO: `/v3/users/009c40702c464facadbc0960b59e505f` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.402818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9b5f4-a0cb-4bc8-9eb6-48b30b826e4c None admin] RBAC: Authorizing `identity:get_user(user_id=009c40702c464facadbc0960b59e505f)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:26.405371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9b5f4-a0cb-4bc8-9eb6-48b30b826e4c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:26.408883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 125/250] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:26 2022] GET /identity/v3/users/009c40702c464facadbc0960b59e505f => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:26.410571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c29c3-13c6-43cc-a531-ac470d36e0ba None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.411058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c29c3-13c6-43cc-a531-ac470d36e0ba None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.411926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c29c3-13c6-43cc-a531-ac470d36e0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.433378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c84c29c3-13c6-43cc-a531-ac470d36e0ba None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:26.434959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c84c29c3-13c6-43cc-a531-ac470d36e0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:26.509452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 126/251] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:26.510468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c29c3-13c6-43cc-a531-ac470d36e0ba None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:26.528859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7792c-cdb2-40dd-87a1-c80b5f427933 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:26.533714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:26.579925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed21fa-4e90-4399-8975-1e3a57945774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:26.580965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed21fa-4e90-4399-8975-1e3a57945774 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.581249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed21fa-4e90-4399-8975-1e3a57945774 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.581527 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed21fa-4e90-4399-8975-1e3a57945774 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.582821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed21fa-4e90-4399-8975-1e3a57945774 None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:26.585266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed21fa-4e90-4399-8975-1e3a57945774 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:26.588171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 126/252] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:26 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:26.589551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193909c-a46e-4061-a8c1-8f54ac5b6a41 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.589837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193909c-a46e-4061-a8c1-8f54ac5b6a41 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.590112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193909c-a46e-4061-a8c1-8f54ac5b6a41 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.591565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 127/253] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:26.592249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193909c-a46e-4061-a8c1-8f54ac5b6a41 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:26.646605 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c29c3-13c6-43cc-a531-ac470d36e0ba None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:26.678537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:26.688339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ea2fe-d2d6-4156-b915-88fc2359cfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:26.689086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ea2fe-d2d6-4156-b915-88fc2359cfcb None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.689487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ea2fe-d2d6-4156-b915-88fc2359cfcb None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.689817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33ea2fe-d2d6-4156-b915-88fc2359cfcb None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.691331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ea2fe-d2d6-4156-b915-88fc2359cfcb None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:26.692950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33ea2fe-d2d6-4156-b915-88fc2359cfcb None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:26.694768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 127/254] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:26 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:26.695742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ea2fe-d2d6-4156-b915-88fc2359cfcb None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:26.737654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193909c-a46e-4061-a8c1-8f54ac5b6a41 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:26.789640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f868a7-f94f-424a-a233-eab256f5a8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:26.790512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f868a7-f94f-424a-a233-eab256f5a8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.790856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f868a7-f94f-424a-a233-eab256f5a8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.791195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f868a7-f94f-424a-a233-eab256f5a8e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.803137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f868a7-f94f-424a-a233-eab256f5a8e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-40f868a7-f94f-424a-a233-eab256f5a8e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:26.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:26.809761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:26.810544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 128/255] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:50:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:26.817280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534369f4-4a1e-41e2-af3d-76dfe69cf26e None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.817672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534369f4-4a1e-41e2-af3d-76dfe69cf26e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.818037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534369f4-4a1e-41e2-af3d-76dfe69cf26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.831502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33ea2fe-d2d6-4156-b915-88fc2359cfcb None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:26.841943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-534369f4-4a1e-41e2-af3d-76dfe69cf26e None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:26.843455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-534369f4-4a1e-41e2-af3d-76dfe69cf26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:26.872044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:26.880022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b74db1-5874-4a44-8368-9f6f9bb2a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:26.880891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b74db1-5874-4a44-8368-9f6f9bb2a7cc None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:26.881306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b74db1-5874-4a44-8368-9f6f9bb2a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:26.881862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b74db1-5874-4a44-8368-9f6f9bb2a7cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:26.883207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b74db1-5874-4a44-8368-9f6f9bb2a7cc None admin] RBAC: Adding query filter params (role.id=cc9d3730bd5c4edcb1897fc5de4e031d, scope.project.id=876fcd5989ee455a89866281d214740e, user.id=2f3d14ad9e6c4fdda59307e744e32b02) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:26.883916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b74db1-5874-4a44-8368-9f6f9bb2a7cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:26.885539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b74db1-5874-4a44-8368-9f6f9bb2a7cc None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:26.892672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 128/256] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:26 2022] GET /identity/v3/role_assignments?user.id=2f3d14ad9e6c4fdda59307e744e32b02&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=cc9d3730bd5c4edcb1897fc5de4e031d => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:26.893766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b74db1-5874-4a44-8368-9f6f9bb2a7cc None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:26.981026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 129/257] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:26.982630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534369f4-4a1e-41e2-af3d-76dfe69cf26e None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:27.047990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:27.123448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b74db1-5874-4a44-8368-9f6f9bb2a7cc None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:27.160202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08bc18-1b7c-45d7-aea2-ac02f5b6bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:27.162707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08bc18-1b7c-45d7-aea2-ac02f5b6bafe None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:27.163168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08bc18-1b7c-45d7-aea2-ac02f5b6bafe None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:27.163627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08bc18-1b7c-45d7-aea2-ac02f5b6bafe None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e/users/009c40702c464facadbc0960b59e505f/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:27.166383 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08bc18-1b7c-45d7-aea2-ac02f5b6bafe None admin] RBAC: Authorizing `identity:create_grant(project_id=876fcd5989ee455a89866281d214740e, user_id=009c40702c464facadbc0960b59e505f, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:27.168125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08bc18-1b7c-45d7-aea2-ac02f5b6bafe None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:27.185834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 129/258] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:50:26 2022] PUT /identity/v3/projects/876fcd5989ee455a89866281d214740e/users/009c40702c464facadbc0960b59e505f/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:27.186913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08bc18-1b7c-45d7-aea2-ac02f5b6bafe None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:27.215379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534369f4-4a1e-41e2-af3d-76dfe69cf26e None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:27.286104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f21ff-ee7b-422d-b382-873d7d543b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:27.286700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f21ff-ee7b-422d-b382-873d7d543b6f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:27.287024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f21ff-ee7b-422d-b382-873d7d543b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:27.287302 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f21ff-ee7b-422d-b382-873d7d543b6f None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:27.288245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f21ff-ee7b-422d-b382-873d7d543b6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:27.288754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f21ff-ee7b-422d-b382-873d7d543b6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:27.290368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f21ff-ee7b-422d-b382-873d7d543b6f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:27.307648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08bc18-1b7c-45d7-aea2-ac02f5b6bafe None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:27.310935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 130/259] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:27.317029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f21ff-ee7b-422d-b382-873d7d543b6f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:27.332806 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:27.349377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8f3de-bf30-49fb-9e08-141934327adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:27.350221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8f3de-bf30-49fb-9e08-141934327adc None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:27.350502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8f3de-bf30-49fb-9e08-141934327adc None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:27.350772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8f3de-bf30-49fb-9e08-141934327adc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:27.364316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8f3de-bf30-49fb-9e08-141934327adc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-f9e8f3de-bf30-49fb-9e08-141934327adc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:27.370097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:27.371067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:27.372046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:27.372658 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:27.372727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 130/260] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:27.379862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8f3de-bf30-49fb-9e08-141934327adc None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:27.494869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8f3de-bf30-49fb-9e08-141934327adc None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:27.555073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4729ea-588d-4576-b24b-874709f32cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:27.555602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4729ea-588d-4576-b24b-874709f32cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:27.555885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4729ea-588d-4576-b24b-874709f32cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:27.556152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4729ea-588d-4576-b24b-874709f32cb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:27.557617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4729ea-588d-4576-b24b-874709f32cb7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:27.557921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4729ea-588d-4576-b24b-874709f32cb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:27.559322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4729ea-588d-4576-b24b-874709f32cb7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:27.568592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 131/261] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:27.571617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f21ff-ee7b-422d-b382-873d7d543b6f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:27.574511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4729ea-588d-4576-b24b-874709f32cb7 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:27.623158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:27.677049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19127f9c-ca5d-4a86-ba1b-8f6c3ef121bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:27.677742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19127f9c-ca5d-4a86-ba1b-8f6c3ef121bd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:27.678054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19127f9c-ca5d-4a86-ba1b-8f6c3ef121bd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:27.678299 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19127f9c-ca5d-4a86-ba1b-8f6c3ef121bd None admin] PATH_INFO: `/v3/users/d03e575f9de24755b9620b237ad12ff8` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:27.679621 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19127f9c-ca5d-4a86-ba1b-8f6c3ef121bd None admin] RBAC: Authorizing `identity:get_user(user_id=d03e575f9de24755b9620b237ad12ff8)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:27.681149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19127f9c-ca5d-4a86-ba1b-8f6c3ef121bd None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:27.682228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4729ea-588d-4576-b24b-874709f32cb7 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:27.682827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 131/262] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:27 2022] GET /identity/v3/users/d03e575f9de24755b9620b237ad12ff8 => generated 283 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:27.688588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19127f9c-ca5d-4a86-ba1b-8f6c3ef121bd None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:27.719197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b533581-01dc-4886-a5ce-f27fc0b8b4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:27.719878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b533581-01dc-4886-a5ce-f27fc0b8b4aa None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:27.720153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b533581-01dc-4886-a5ce-f27fc0b8b4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:27.720469 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b533581-01dc-4886-a5ce-f27fc0b8b4aa None admin] PATH_INFO: `/v3/users/d03e575f9de24755b9620b237ad12ff8` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:27.721722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b533581-01dc-4886-a5ce-f27fc0b8b4aa None admin] RBAC: Authorizing `identity:get_user(user_id=d03e575f9de24755b9620b237ad12ff8)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:27.723238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b533581-01dc-4886-a5ce-f27fc0b8b4aa None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:27.725332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 132/263] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:27 2022] GET /identity/v3/users/d03e575f9de24755b9620b237ad12ff8 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:27.731191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b533581-01dc-4886-a5ce-f27fc0b8b4aa None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:27.894696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b533581-01dc-4886-a5ce-f27fc0b8b4aa None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:27.916865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:27.925120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:27.952407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19127f9c-ca5d-4a86-ba1b-8f6c3ef121bd None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:27.985871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e3188-8a5c-4333-a7cc-1f06c27d77cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:27.986934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e3188-8a5c-4333-a7cc-1f06c27d77cd None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:27.987795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e3188-8a5c-4333-a7cc-1f06c27d77cd None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:27.988280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e3188-8a5c-4333-a7cc-1f06c27d77cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:28.014223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a9c08-59c3-48df-8675-30fd51fcf31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:28.014889 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a9c08-59c3-48df-8675-30fd51fcf31b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:28.015145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a9c08-59c3-48df-8675-30fd51fcf31b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:28.015417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a9c08-59c3-48df-8675-30fd51fcf31b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:28.133728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a9c08-59c3-48df-8675-30fd51fcf31b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:28.135151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a9c08-59c3-48df-8675-30fd51fcf31b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:28.143416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e3188-8a5c-4333-a7cc-1f06c27d77cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:28.145585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e3188-8a5c-4333-a7cc-1f06c27d77cd None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:28.248458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 133/264] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 11:50:28.263327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e3188-8a5c-4333-a7cc-1f06c27d77cd None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:28.277306 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:28.285347 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:28.289402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 132/265] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 11:50:28.295256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a9c08-59c3-48df-8675-30fd51fcf31b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:28.407654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a9c08-59c3-48df-8675-30fd51fcf31b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:28.445766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd314a5-32fc-4d12-b4ce-054ee978a7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:28.446869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd314a5-32fc-4d12-b4ce-054ee978a7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:28.447378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd314a5-32fc-4d12-b4ce-054ee978a7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:28.447840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd314a5-32fc-4d12-b4ce-054ee978a7e9 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:28.449238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd314a5-32fc-4d12-b4ce-054ee978a7e9 None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:28.450916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd314a5-32fc-4d12-b4ce-054ee978a7e9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:28.453154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 133/266] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:28 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:28.459250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd314a5-32fc-4d12-b4ce-054ee978a7e9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:28.519318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e3188-8a5c-4333-a7cc-1f06c27d77cd None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:28.520686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:28.582308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd314a5-32fc-4d12-b4ce-054ee978a7e9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:28.607914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3169b5-8ea0-4994-981e-f9cbca7c0f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:28.608629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3169b5-8ea0-4994-981e-f9cbca7c0f09 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:28.608909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3169b5-8ea0-4994-981e-f9cbca7c0f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:28.609180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3169b5-8ea0-4994-981e-f9cbca7c0f09 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:28.610640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3169b5-8ea0-4994-981e-f9cbca7c0f09 None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:28.612112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3169b5-8ea0-4994-981e-f9cbca7c0f09 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:28.614239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 134/267] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:28 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:28.622755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3169b5-8ea0-4994-981e-f9cbca7c0f09 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:28.674602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400b0a15-5087-4679-b179-79ac3da1d651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:28.675157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b0a15-5087-4679-b179-79ac3da1d651 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:28.675442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b0a15-5087-4679-b179-79ac3da1d651 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:28.675719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400b0a15-5087-4679-b179-79ac3da1d651 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:28.676566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b0a15-5087-4679-b179-79ac3da1d651 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=876fcd5989ee455a89866281d214740e, user.id=d03e575f9de24755b9620b237ad12ff8) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:28.676877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b0a15-5087-4679-b179-79ac3da1d651 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:28.678367 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400b0a15-5087-4679-b179-79ac3da1d651 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:28.698223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 134/268] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:28 2022] GET /identity/v3/role_assignments?user.id=d03e575f9de24755b9620b237ad12ff8&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:28.760686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:28.792438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3169b5-8ea0-4994-981e-f9cbca7c0f09 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:28.842569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b18eb-8d4c-4736-8da9-b1e42189c12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:28.843283 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b18eb-8d4c-4736-8da9-b1e42189c12d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:28.843602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b18eb-8d4c-4736-8da9-b1e42189c12d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:28.843878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b18eb-8d4c-4736-8da9-b1e42189c12d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:28.844732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b18eb-8d4c-4736-8da9-b1e42189c12d None admin] RBAC: Adding query filter params (role.id=8e13cd96bb124d52b5c8883fe960424f, scope.project.id=876fcd5989ee455a89866281d214740e, user.id=d03e575f9de24755b9620b237ad12ff8) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:28.845041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b18eb-8d4c-4736-8da9-b1e42189c12d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:28.853723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b18eb-8d4c-4736-8da9-b1e42189c12d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:28.860858 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 135/269] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:28 2022] GET /identity/v3/role_assignments?user.id=d03e575f9de24755b9620b237ad12ff8&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=8e13cd96bb124d52b5c8883fe960424f => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:29.013063 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad394382-7a69-4991-8443-72ccf35fee18 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.013368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad394382-7a69-4991-8443-72ccf35fee18 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.013545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad394382-7a69-4991-8443-72ccf35fee18 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.014981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 135/270] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:29.021249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5fdef-50d9-40b0-87aa-f9071157f11a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.021535 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5fdef-50d9-40b0-87aa-f9071157f11a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.021816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5fdef-50d9-40b0-87aa-f9071157f11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.049523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-35b5fdef-50d9-40b0-87aa-f9071157f11a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:29.050783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-35b5fdef-50d9-40b0-87aa-f9071157f11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:29.135600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 136/271] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:29.162908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad394382-7a69-4991-8443-72ccf35fee18 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:29.274697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bfe40-e6b5-4228-b3d9-8f622b5cbf39 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.275197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bfe40-e6b5-4228-b3d9-8f622b5cbf39 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.275197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bfe40-e6b5-4228-b3d9-8f622b5cbf39 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.276949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 137/272] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:29.286259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127ca6e-95d9-42be-b7ce-cda0b347f813 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.286579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127ca6e-95d9-42be-b7ce-cda0b347f813 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.286832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6127ca6e-95d9-42be-b7ce-cda0b347f813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.316725 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-6127ca6e-95d9-42be-b7ce-cda0b347f813 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:29.318649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6127ca6e-95d9-42be-b7ce-cda0b347f813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:29.323004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad394382-7a69-4991-8443-72ccf35fee18 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:29.357686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:29.377092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe8fdf-f86e-4353-8976-556fdc1f1796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:29.377856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe8fdf-f86e-4353-8976-556fdc1f1796 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.378144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe8fdf-f86e-4353-8976-556fdc1f1796 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.378602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe8fdf-f86e-4353-8976-556fdc1f1796 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.379783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe8fdf-f86e-4353-8976-556fdc1f1796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:29.381514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe8fdf-f86e-4353-8976-556fdc1f1796 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:29.383348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 136/273] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:29.389019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe8fdf-f86e-4353-8976-556fdc1f1796 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:29.407929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 138/274] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:29.437037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127ca6e-95d9-42be-b7ce-cda0b347f813 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:29.439798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:29.512890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe8fdf-f86e-4353-8976-556fdc1f1796 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:29.553335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c421dd3c-2dbd-41c3-942c-019b7898bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:29.553910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421dd3c-2dbd-41c3-942c-019b7898bcee None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.554187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421dd3c-2dbd-41c3-942c-019b7898bcee None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.554691 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c421dd3c-2dbd-41c3-942c-019b7898bcee None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.555331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421dd3c-2dbd-41c3-942c-019b7898bcee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:29.555856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6127ca6e-95d9-42be-b7ce-cda0b347f813 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:29.556829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c421dd3c-2dbd-41c3-942c-019b7898bcee None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:29.588336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 137/275] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:50:29 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:29.588336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0605ae-8431-4808-9680-ee450f069812 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.588336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0605ae-8431-4808-9680-ee450f069812 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.588336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0605ae-8431-4808-9680-ee450f069812 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.588336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 138/276] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:29.591110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e4cf-a538-4bd6-9e43-c37c1c9486ef None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.591400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e4cf-a538-4bd6-9e43-c37c1c9486ef None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.591920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9e4cf-a538-4bd6-9e43-c37c1c9486ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.605973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178252d1-5e60-4cae-9e51-2d310c217be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:29.606767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178252d1-5e60-4cae-9e51-2d310c217be5 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.607059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178252d1-5e60-4cae-9e51-2d310c217be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.607307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178252d1-5e60-4cae-9e51-2d310c217be5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.614706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-13d9e4cf-a538-4bd6-9e43-c37c1c9486ef None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:29.617164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-13d9e4cf-a538-4bd6-9e43-c37c1c9486ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:29.621360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178252d1-5e60-4cae-9e51-2d310c217be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-178252d1-5e60-4cae-9e51-2d310c217be5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:29.626741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:29.627810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:29.629074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 139/277] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:29.634258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178252d1-5e60-4cae-9e51-2d310c217be5 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:29.685353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:29.714481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:29.728323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 139/278] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:29.745692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9e4cf-a538-4bd6-9e43-c37c1c9486ef None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:29.767152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178252d1-5e60-4cae-9e51-2d310c217be5 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:29.815255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9fe58-f05d-4839-97fc-d29730ff8edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:29.815914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9fe58-f05d-4839-97fc-d29730ff8edb None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.816189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9fe58-f05d-4839-97fc-d29730ff8edb None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.816585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9fe58-f05d-4839-97fc-d29730ff8edb None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.817318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9fe58-f05d-4839-97fc-d29730ff8edb None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:29.817616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9fe58-f05d-4839-97fc-d29730ff8edb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:29.819145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a9fe58-f05d-4839-97fc-d29730ff8edb None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:29.829187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 140/279] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:29.834994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9fe58-f05d-4839-97fc-d29730ff8edb None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:29.892386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9e4cf-a538-4bd6-9e43-c37c1c9486ef None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:29.937664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644314b1-2b97-4569-8dd0-ded2ec0115e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:29.939128 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644314b1-2b97-4569-8dd0-ded2ec0115e0 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:29.939557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644314b1-2b97-4569-8dd0-ded2ec0115e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:29.939996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644314b1-2b97-4569-8dd0-ded2ec0115e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:29.954588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644314b1-2b97-4569-8dd0-ded2ec0115e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-644314b1-2b97-4569-8dd0-ded2ec0115e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:29.959795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:29.960826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:29.961630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:29.962745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 140/280] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:29.968442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644314b1-2b97-4569-8dd0-ded2ec0115e0 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:29.974803 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a9fe58-f05d-4839-97fc-d29730ff8edb None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:30.008014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:30.016911 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:30.024548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43898f4-e630-4e70-bbde-67b8417664d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:30.025622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43898f4-e630-4e70-bbde-67b8417664d4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:30.025936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43898f4-e630-4e70-bbde-67b8417664d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:30.026468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43898f4-e630-4e70-bbde-67b8417664d4 None admin] PATH_INFO: `/v3/users/7f9bc4a3b3d84886b2586d45cb4e461b` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:30.049955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43898f4-e630-4e70-bbde-67b8417664d4 None admin] RBAC: Authorizing `identity:get_user(user_id=7f9bc4a3b3d84886b2586d45cb4e461b)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:30.051468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43898f4-e630-4e70-bbde-67b8417664d4 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:30.056321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 141/281] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:29 2022] GET /identity/v3/users/7f9bc4a3b3d84886b2586d45cb4e461b => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:30.060280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43898f4-e630-4e70-bbde-67b8417664d4 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:30.111155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644314b1-2b97-4569-8dd0-ded2ec0115e0 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:30.150033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05a5cc-6b85-4280-aada-9922aaedd8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:30.150690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05a5cc-6b85-4280-aada-9922aaedd8ae None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:30.150968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05a5cc-6b85-4280-aada-9922aaedd8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:30.151240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05a5cc-6b85-4280-aada-9922aaedd8ae None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:30.151998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05a5cc-6b85-4280-aada-9922aaedd8ae None admin] RBAC: Adding query filter params (name=reader) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:30.152324 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05a5cc-6b85-4280-aada-9922aaedd8ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:30.153786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05a5cc-6b85-4280-aada-9922aaedd8ae None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:30.163472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 141/282] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:30.184144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05a5cc-6b85-4280-aada-9922aaedd8ae None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:30.189054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43898f4-e630-4e70-bbde-67b8417664d4 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:30.227132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e661e52d-60b7-40d2-9b65-f931d1e34d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:30.228088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661e52d-60b7-40d2-9b65-f931d1e34d5f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:30.228577 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661e52d-60b7-40d2-9b65-f931d1e34d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:30.228995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e661e52d-60b7-40d2-9b65-f931d1e34d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:30.230058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661e52d-60b7-40d2-9b65-f931d1e34d5f None admin] RBAC: Adding query filter params (role.id=4f5140de9df4440a8fb63d93249d14d1, user.id=7f9bc4a3b3d84886b2586d45cb4e461b, scope.system=all) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:30.230514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661e52d-60b7-40d2-9b65-f931d1e34d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:30.232159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e661e52d-60b7-40d2-9b65-f931d1e34d5f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:30.239006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 142/283] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Aug 1 11:50:30 2022] GET /identity/v3/role_assignments?user.id=7f9bc4a3b3d84886b2586d45cb4e461b&scope.system=all&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:30.248985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228b419-c576-4ca0-b4e4-d55e763a3bd6 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:30.249810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228b419-c576-4ca0-b4e4-d55e763a3bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:30.250227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a228b419-c576-4ca0-b4e4-d55e763a3bd6 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:30.252166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 143/284] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:30.258025 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc04af-9041-4cb0-835f-d2a2dcd0c6c7 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:30.258621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc04af-9041-4cb0-835f-d2a2dcd0c6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:30.259054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc04af-9041-4cb0-835f-d2a2dcd0c6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:30.296321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-84cc04af-9041-4cb0-835f-d2a2dcd0c6c7 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:30.297572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-84cc04af-9041-4cb0-835f-d2a2dcd0c6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:30.329631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:30.333439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05a5cc-6b85-4280-aada-9922aaedd8ae None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:30.388417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:30.415962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc544c9-6140-44eb-bfb3-f5396d911a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:30.417577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc544c9-6140-44eb-bfb3-f5396d911a85 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:30.418179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc544c9-6140-44eb-bfb3-f5396d911a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:30.418601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc544c9-6140-44eb-bfb3-f5396d911a85 None admin] PATH_INFO: `/v3/users/2f3d14ad9e6c4fdda59307e744e32b02` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:30.420241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc544c9-6140-44eb-bfb3-f5396d911a85 None admin] RBAC: Authorizing `identity:get_user(user_id=2f3d14ad9e6c4fdda59307e744e32b02)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:30.422239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc544c9-6140-44eb-bfb3-f5396d911a85 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:30.427536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 142/285] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:30 2022] GET /identity/v3/users/2f3d14ad9e6c4fdda59307e744e32b02 => generated 297 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:30.432564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc544c9-6140-44eb-bfb3-f5396d911a85 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:30.447775 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 144/286] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:30.482901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc04af-9041-4cb0-835f-d2a2dcd0c6c7 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:30.569122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc544c9-6140-44eb-bfb3-f5396d911a85 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:30.606797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36ee6f-d4d4-4744-9ecd-9daff7b1beaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:30.607456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36ee6f-d4d4-4744-9ecd-9daff7b1beaf None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:30.607733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36ee6f-d4d4-4744-9ecd-9daff7b1beaf None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:30.608004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36ee6f-d4d4-4744-9ecd-9daff7b1beaf None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:30.609210 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36ee6f-d4d4-4744-9ecd-9daff7b1beaf None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:30.610680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36ee6f-d4d4-4744-9ecd-9daff7b1beaf None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:30.614039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 143/287] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:30 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:30.619775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36ee6f-d4d4-4744-9ecd-9daff7b1beaf None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:30.668931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc04af-9041-4cb0-835f-d2a2dcd0c6c7 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:30.686875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:30.743766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453c7d5-cf83-443b-ac38-4ef56ccd51f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:30.744462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453c7d5-cf83-443b-ac38-4ef56ccd51f9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:30.744744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453c7d5-cf83-443b-ac38-4ef56ccd51f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:30.745017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1453c7d5-cf83-443b-ac38-4ef56ccd51f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:30.759304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1453c7d5-cf83-443b-ac38-4ef56ccd51f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-1453c7d5-cf83-443b-ac38-4ef56ccd51f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:30.765437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:30.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:30.767095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 145/288] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:30.773601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453c7d5-cf83-443b-ac38-4ef56ccd51f9 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:30.833311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:30.885402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1453c7d5-cf83-443b-ac38-4ef56ccd51f9 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:30.886476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36ee6f-d4d4-4744-9ecd-9daff7b1beaf None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:30.924309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3b57c-ea2e-4990-9bfa-aae6ed0c79a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:30.924909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3b57c-ea2e-4990-9bfa-aae6ed0c79a0 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:30.925193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3b57c-ea2e-4990-9bfa-aae6ed0c79a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:30.925464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3b57c-ea2e-4990-9bfa-aae6ed0c79a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:30.926194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3b57c-ea2e-4990-9bfa-aae6ed0c79a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:30.926498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3b57c-ea2e-4990-9bfa-aae6ed0c79a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:30.927903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3b57c-ea2e-4990-9bfa-aae6ed0c79a0 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:30.942949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 146/289] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:30.944271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3b57c-ea2e-4990-9bfa-aae6ed0c79a0 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:30.993142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4021f33-bc05-4c6c-9cb4-30fb0122ad27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:30.993830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4021f33-bc05-4c6c-9cb4-30fb0122ad27 None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:30.994115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4021f33-bc05-4c6c-9cb4-30fb0122ad27 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:30.994391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4021f33-bc05-4c6c-9cb4-30fb0122ad27 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e/users/2f3d14ad9e6c4fdda59307e744e32b02/roles/cc9d3730bd5c4edcb1897fc5de4e031d` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:30.998021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4021f33-bc05-4c6c-9cb4-30fb0122ad27 None admin] RBAC: Authorizing `identity:create_grant(project_id=876fcd5989ee455a89866281d214740e, user_id=2f3d14ad9e6c4fdda59307e744e32b02, role_id=cc9d3730bd5c4edcb1897fc5de4e031d)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:30.999489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4021f33-bc05-4c6c-9cb4-30fb0122ad27 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:31.020149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 144/290] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:50:30 2022] PUT /identity/v3/projects/876fcd5989ee455a89866281d214740e/users/2f3d14ad9e6c4fdda59307e744e32b02/roles/cc9d3730bd5c4edcb1897fc5de4e031d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:31.095318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:31.099854 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3b57c-ea2e-4990-9bfa-aae6ed0c79a0 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:31.151430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0227c-e536-49bf-ae26-288acb7e6687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:31.152098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0227c-e536-49bf-ae26-288acb7e6687 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:31.152394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0227c-e536-49bf-ae26-288acb7e6687 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:31.152671 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0227c-e536-49bf-ae26-288acb7e6687 None admin] PATH_INFO: `/v3/users/009c40702c464facadbc0960b59e505f` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:31.153881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0227c-e536-49bf-ae26-288acb7e6687 None admin] RBAC: Authorizing `identity:get_user(user_id=009c40702c464facadbc0960b59e505f)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:31.155340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0227c-e536-49bf-ae26-288acb7e6687 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:31.157068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 147/291] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:30 2022] GET /identity/v3/users/009c40702c464facadbc0960b59e505f => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:31.162809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4021f33-bc05-4c6c-9cb4-30fb0122ad27 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:31.289486 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:31.299416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4021f33-bc05-4c6c-9cb4-30fb0122ad27 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:31.338584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fd3e6-ed8a-43ab-8bb7-830ddf33d615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:31.339282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fd3e6-ed8a-43ab-8bb7-830ddf33d615 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:31.339558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fd3e6-ed8a-43ab-8bb7-830ddf33d615 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:31.339827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fd3e6-ed8a-43ab-8bb7-830ddf33d615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:31.415395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fd3e6-ed8a-43ab-8bb7-830ddf33d615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:31.416908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fd3e6-ed8a-43ab-8bb7-830ddf33d615 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:31.548019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 145/292] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:31.548586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:31.553731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0227c-e536-49bf-ae26-288acb7e6687 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:31.711343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0227c-e536-49bf-ae26-288acb7e6687 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:31.762329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeba81f-b5a6-4789-8758-de9fd472e60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:31.763076 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeba81f-b5a6-4789-8758-de9fd472e60d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:31.763368 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeba81f-b5a6-4789-8758-de9fd472e60d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:31.763643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeba81f-b5a6-4789-8758-de9fd472e60d None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:31.765044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeba81f-b5a6-4789-8758-de9fd472e60d None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:31.766554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeba81f-b5a6-4789-8758-de9fd472e60d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:31.769818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 148/293] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:31 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:31.774885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fd3e6-ed8a-43ab-8bb7-830ddf33d615 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:31.942239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fd3e6-ed8a-43ab-8bb7-830ddf33d615 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:31.989630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b13a84-210b-4cdb-a59c-a12415cc6561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:31.990259 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b13a84-210b-4cdb-a59c-a12415cc6561 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:31.990536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b13a84-210b-4cdb-a59c-a12415cc6561 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:31.990806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b13a84-210b-4cdb-a59c-a12415cc6561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:31.991641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b13a84-210b-4cdb-a59c-a12415cc6561 None admin] RBAC: Adding query filter params (role.id=4f5140de9df4440a8fb63d93249d14d1, scope.project.id=876fcd5989ee455a89866281d214740e, user.id=009c40702c464facadbc0960b59e505f) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:31.991948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b13a84-210b-4cdb-a59c-a12415cc6561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:31.993434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b13a84-210b-4cdb-a59c-a12415cc6561 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:32.005156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 146/294] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:31 2022] GET /identity/v3/role_assignments?user.id=009c40702c464facadbc0960b59e505f&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:32.188841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ace2f-be11-41f2-98a2-159581dae836 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.188841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ace2f-be11-41f2-98a2-159581dae836 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.188841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ace2f-be11-41f2-98a2-159581dae836 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.188841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 149/295] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:32.195224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8629b-8f7c-437d-a892-4a3d4619ff86 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.195733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8629b-8f7c-437d-a892-4a3d4619ff86 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.195840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8629b-8f7c-437d-a892-4a3d4619ff86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.202548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298d326-1d23-413e-b0e4-eb26176f80da None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.202827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298d326-1d23-413e-b0e4-eb26176f80da None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.203113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298d326-1d23-413e-b0e4-eb26176f80da None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.204538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 150/296] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:32.210073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b6b74-69b5-4ec7-a32b-bbd9366c3386 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.210427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b6b74-69b5-4ec7-a32b-bbd9366c3386 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.210680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b6b74-69b5-4ec7-a32b-bbd9366c3386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.226791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-31a8629b-8f7c-437d-a892-4a3d4619ff86 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:32.228161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-31a8629b-8f7c-437d-a892-4a3d4619ff86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:32.231209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c08b6b74-69b5-4ec7-a32b-bbd9366c3386 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:32.232471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c08b6b74-69b5-4ec7-a32b-bbd9366c3386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:32.290770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:32.322991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:32.342013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 147/297] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:32.357652 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 151/298] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:32.367134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8629b-8f7c-437d-a892-4a3d4619ff86 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:32.384206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b6b74-69b5-4ec7-a32b-bbd9366c3386 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:32.492995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8629b-8f7c-437d-a892-4a3d4619ff86 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:32.514166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b6b74-69b5-4ec7-a32b-bbd9366c3386 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:32.542108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd131f8-ae2a-449d-8b57-70af7d63b2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:32.542772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd131f8-ae2a-449d-8b57-70af7d63b2eb None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.543058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd131f8-ae2a-449d-8b57-70af7d63b2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.543331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd131f8-ae2a-449d-8b57-70af7d63b2eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.563420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd131f8-ae2a-449d-8b57-70af7d63b2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-0fd131f8-ae2a-449d-8b57-70af7d63b2eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:32.570301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:32.571230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:32.572037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:32.573848 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9a5a6-6140-4d49-827b-363828261cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:32.575051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9a5a6-6140-4d49-827b-363828261cbc None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.575526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9a5a6-6140-4d49-827b-363828261cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.575982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9a5a6-6140-4d49-827b-363828261cbc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.576574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 148/299] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:32.577541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd131f8-ae2a-449d-8b57-70af7d63b2eb None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:32.578237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9a5a6-6140-4d49-827b-363828261cbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:32.580006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9a5a6-6140-4d49-827b-363828261cbc None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:32.583200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 152/300] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:32.590035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9a5a6-6140-4d49-827b-363828261cbc None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:32.622890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:32.632714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:32.683892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd131f8-ae2a-449d-8b57-70af7d63b2eb None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:32.700491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9a5a6-6140-4d49-827b-363828261cbc None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:32.721321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb545b3-02d5-4cbb-a8a1-631d3d4fe9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:32.721956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb545b3-02d5-4cbb-a8a1-631d3d4fe9bd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.722237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb545b3-02d5-4cbb-a8a1-631d3d4fe9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.722513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb545b3-02d5-4cbb-a8a1-631d3d4fe9bd None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.723272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb545b3-02d5-4cbb-a8a1-631d3d4fe9bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:32.723554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb545b3-02d5-4cbb-a8a1-631d3d4fe9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:32.725001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb545b3-02d5-4cbb-a8a1-631d3d4fe9bd None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:32.734809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 149/301] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:32.740961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb545b3-02d5-4cbb-a8a1-631d3d4fe9bd None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:32.744949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89888081-9ee4-4e80-aec0-4d66c99c305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:32.745927 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89888081-9ee4-4e80-aec0-4d66c99c305c None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.746478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89888081-9ee4-4e80-aec0-4d66c99c305c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.746965 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89888081-9ee4-4e80-aec0-4d66c99c305c None admin] PATH_INFO: `/v3/groups` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.747853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89888081-9ee4-4e80-aec0-4d66c99c305c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:32.749565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89888081-9ee4-4e80-aec0-4d66c99c305c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:32.776902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 153/302] 10.222.0.49 () {60 vars in 1134 bytes} [Mon Aug 1 11:50:32 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 11:50:32.797824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48c132-2b2c-4dd2-a9ad-4b3035e235c8 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.798206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48c132-2b2c-4dd2-a9ad-4b3035e235c8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.798390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48c132-2b2c-4dd2-a9ad-4b3035e235c8 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.799983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 154/303] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:32.805606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73874706-bcae-4933-ab33-45cbae37f000 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.805893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73874706-bcae-4933-ab33-45cbae37f000 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.806184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73874706-bcae-4933-ab33-45cbae37f000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.838815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-73874706-bcae-4933-ab33-45cbae37f000 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:32.840869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-73874706-bcae-4933-ab33-45cbae37f000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:32.878348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb545b3-02d5-4cbb-a8a1-631d3d4fe9bd None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:32.895888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:32.924952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 155/304] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:32.939643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e7ea8-ec51-45c2-97fd-5b8121b7364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:32.940899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e7ea8-ec51-45c2-97fd-5b8121b7364d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:32.941364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e7ea8-ec51-45c2-97fd-5b8121b7364d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:32.941817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e7ea8-ec51-45c2-97fd-5b8121b7364d None admin] PATH_INFO: `/v3/users/49216cd01efe4743b774907c083ad0aa` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:32.947998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73874706-bcae-4933-ab33-45cbae37f000 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:32.960579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e7ea8-ec51-45c2-97fd-5b8121b7364d None admin] RBAC: Authorizing `identity:get_user(user_id=49216cd01efe4743b774907c083ad0aa)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:32.963165 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e7ea8-ec51-45c2-97fd-5b8121b7364d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:32.966318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 150/305] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:32 2022] GET /identity/v3/users/49216cd01efe4743b774907c083ad0aa => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:32.972505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e7ea8-ec51-45c2-97fd-5b8121b7364d None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:32.982784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:33.073546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73874706-bcae-4933-ab33-45cbae37f000 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:33.105409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e7ea8-ec51-45c2-97fd-5b8121b7364d None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:33.121476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dedc3d-126d-4a4d-8df6-a6af12c73b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:33.122886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dedc3d-126d-4a4d-8df6-a6af12c73b02 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:33.123340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dedc3d-126d-4a4d-8df6-a6af12c73b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:33.123991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dedc3d-126d-4a4d-8df6-a6af12c73b02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:33.139512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dedc3d-126d-4a4d-8df6-a6af12c73b02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:33.145114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2730d-426a-4a2c-a00e-15a7ba17424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:33.145862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2730d-426a-4a2c-a00e-15a7ba17424c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-06dedc3d-126d-4a4d-8df6-a6af12c73b02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:33.146078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:33.147091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:33.148181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:33.148962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2730d-426a-4a2c-a00e-15a7ba17424c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:33.149176 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2730d-426a-4a2c-a00e-15a7ba17424c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:33.150083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2730d-426a-4a2c-a00e-15a7ba17424c None admin] RBAC: Adding query filter params (role.id=cc9d3730bd5c4edcb1897fc5de4e031d, user.id=49216cd01efe4743b774907c083ad0aa, scope.system=all) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:33.150541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2730d-426a-4a2c-a00e-15a7ba17424c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:33.151115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 156/306] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:33.156125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2730d-426a-4a2c-a00e-15a7ba17424c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:33.157916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dedc3d-126d-4a4d-8df6-a6af12c73b02 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:33.160510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 151/307] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Aug 1 11:50:32 2022] GET /identity/v3/role_assignments?user.id=49216cd01efe4743b774907c083ad0aa&scope.system=all&role.id=cc9d3730bd5c4edcb1897fc5de4e031d => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:33.255160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:33.276095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dedc3d-126d-4a4d-8df6-a6af12c73b02 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:33.315658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14202ed7-f503-4e6f-8e4a-512aacd5eaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:33.316324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14202ed7-f503-4e6f-8e4a-512aacd5eaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:33.316613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14202ed7-f503-4e6f-8e4a-512aacd5eaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:33.316891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14202ed7-f503-4e6f-8e4a-512aacd5eaa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:33.317631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14202ed7-f503-4e6f-8e4a-512aacd5eaa6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:33.317941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14202ed7-f503-4e6f-8e4a-512aacd5eaa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:33.319369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14202ed7-f503-4e6f-8e4a-512aacd5eaa6 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:33.329244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 157/308] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:33.334933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2730d-426a-4a2c-a00e-15a7ba17424c None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:33.377092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:33.457386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2730d-426a-4a2c-a00e-15a7ba17424c None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:33.493836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be35076c-778b-4d24-b8cb-c2a5578ac3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:33.494491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35076c-778b-4d24-b8cb-c2a5578ac3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:33.494764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35076c-778b-4d24-b8cb-c2a5578ac3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:33.495031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be35076c-778b-4d24-b8cb-c2a5578ac3d9 None admin] PATH_INFO: `/v3/users/7f9bc4a3b3d84886b2586d45cb4e461b` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:33.496239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35076c-778b-4d24-b8cb-c2a5578ac3d9 None admin] RBAC: Authorizing `identity:get_user(user_id=7f9bc4a3b3d84886b2586d45cb4e461b)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:33.497709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be35076c-778b-4d24-b8cb-c2a5578ac3d9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:33.499681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 152/309] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:33 2022] GET /identity/v3/users/7f9bc4a3b3d84886b2586d45cb4e461b => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:33.504988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14202ed7-f503-4e6f-8e4a-512aacd5eaa6 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:33.617408 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14202ed7-f503-4e6f-8e4a-512aacd5eaa6 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:33.652832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f268f3-ae75-4dbb-a8b6-5f8605952f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:33.653477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f268f3-ae75-4dbb-a8b6-5f8605952f9b None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:33.653750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f268f3-ae75-4dbb-a8b6-5f8605952f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:33.654020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f268f3-ae75-4dbb-a8b6-5f8605952f9b None admin] PATH_INFO: `/v3/system/users/7f9bc4a3b3d84886b2586d45cb4e461b/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:33.654622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ee7a0-a48e-4d68-ba2b-b7dd73356c7e None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:33.654906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ee7a0-a48e-4d68-ba2b-b7dd73356c7e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:33.655174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ee7a0-a48e-4d68-ba2b-b7dd73356c7e None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:33.655521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f268f3-ae75-4dbb-a8b6-5f8605952f9b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7f9bc4a3b3d84886b2586d45cb4e461b, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:33.656808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 153/310] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:33.656925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f268f3-ae75-4dbb-a8b6-5f8605952f9b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:33.662807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28cbfe-0f88-429b-a349-d5226ac83c21 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:33.663119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28cbfe-0f88-429b-a349-d5226ac83c21 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:33.663427 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d28cbfe-0f88-429b-a349-d5226ac83c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:33.670092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 158/311] 10.222.0.49 () {58 vars in 1334 bytes} [Mon Aug 1 11:50:33 2022] PUT /identity/v3/system/users/7f9bc4a3b3d84886b2586d45cb4e461b/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:33.688017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-1d28cbfe-0f88-429b-a349-d5226ac83c21 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:33.689809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1d28cbfe-0f88-429b-a349-d5226ac83c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:33.770489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 154/312] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:33.794775 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f268f3-ae75-4dbb-a8b6-5f8605952f9b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:33.807817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:33.911806 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f268f3-ae75-4dbb-a8b6-5f8605952f9b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:33.947315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5527f-fa5d-447c-8f32-b325004a78ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:33.947986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5527f-fa5d-447c-8f32-b325004a78ea None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:33.948278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5527f-fa5d-447c-8f32-b325004a78ea None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:33.948586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5527f-fa5d-447c-8f32-b325004a78ea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:33.959601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5527f-fa5d-447c-8f32-b325004a78ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-9ff5527f-fa5d-447c-8f32-b325004a78ea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:33.965313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:33.966199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:33.967211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 159/313] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:33.974240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28cbfe-0f88-429b-a349-d5226ac83c21 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:34.005319 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:34.101780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d28cbfe-0f88-429b-a349-d5226ac83c21 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:34.137722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba053a4-3e6d-465f-913f-5fd08c7071a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:34.138357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba053a4-3e6d-465f-913f-5fd08c7071a1 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:34.138635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba053a4-3e6d-465f-913f-5fd08c7071a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:34.138906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba053a4-3e6d-465f-913f-5fd08c7071a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:34.139633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba053a4-3e6d-465f-913f-5fd08c7071a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:34.139935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba053a4-3e6d-465f-913f-5fd08c7071a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:34.141401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba053a4-3e6d-465f-913f-5fd08c7071a1 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:34.150869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 155/314] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:34.157606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5527f-fa5d-447c-8f32-b325004a78ea None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:34.226219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:34.261052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5527f-fa5d-447c-8f32-b325004a78ea None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:34.295693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025ab7f-8e30-4462-adb8-3b5c0b086515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:34.296376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025ab7f-8e30-4462-adb8-3b5c0b086515 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:34.296655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025ab7f-8e30-4462-adb8-3b5c0b086515 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:34.296998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025ab7f-8e30-4462-adb8-3b5c0b086515 None admin] PATH_INFO: `/v3/users/2f3d14ad9e6c4fdda59307e744e32b02` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:34.298110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025ab7f-8e30-4462-adb8-3b5c0b086515 None admin] RBAC: Authorizing `identity:get_user(user_id=2f3d14ad9e6c4fdda59307e744e32b02)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:34.299573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d025ab7f-8e30-4462-adb8-3b5c0b086515 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:34.301527 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 160/315] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:34 2022] GET /identity/v3/users/2f3d14ad9e6c4fdda59307e744e32b02 => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:34.308764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba053a4-3e6d-465f-913f-5fd08c7071a1 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:34.417458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:34.426811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba053a4-3e6d-465f-913f-5fd08c7071a1 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:34.462873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad6979-aaad-41d5-8c5a-7b628e145fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:34.463500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad6979-aaad-41d5-8c5a-7b628e145fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:34.463778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad6979-aaad-41d5-8c5a-7b628e145fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:34.464050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad6979-aaad-41d5-8c5a-7b628e145fa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:34.532947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad6979-aaad-41d5-8c5a-7b628e145fa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:34.534332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ad6979-aaad-41d5-8c5a-7b628e145fa1 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:34.624884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:34.634950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 156/316] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 11:50:34.640787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025ab7f-8e30-4462-adb8-3b5c0b086515 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:34.757622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025ab7f-8e30-4462-adb8-3b5c0b086515 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:34.792443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31c903-d4e9-44fb-ac30-762339256aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:34.793013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31c903-d4e9-44fb-ac30-762339256aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:34.793290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31c903-d4e9-44fb-ac30-762339256aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:34.793559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31c903-d4e9-44fb-ac30-762339256aa2 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:34.794796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31c903-d4e9-44fb-ac30-762339256aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:34.796326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31c903-d4e9-44fb-ac30-762339256aa2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:34.798301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 161/317] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:34 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:34.805450 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad6979-aaad-41d5-8c5a-7b628e145fa1 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:34.911913 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ad6979-aaad-41d5-8c5a-7b628e145fa1 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:34.946741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277c788-8be7-414d-9124-dfd1ffdc4934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:34.951420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277c788-8be7-414d-9124-dfd1ffdc4934 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:34.951420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277c788-8be7-414d-9124-dfd1ffdc4934 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:34.951420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277c788-8be7-414d-9124-dfd1ffdc4934 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:34.951420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277c788-8be7-414d-9124-dfd1ffdc4934 None admin] RBAC: Adding query filter params (role.id=cc9d3730bd5c4edcb1897fc5de4e031d, scope.project.id=876fcd5989ee455a89866281d214740e, user.id=2f3d14ad9e6c4fdda59307e744e32b02) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:34.951420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277c788-8be7-414d-9124-dfd1ffdc4934 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:34.951420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b277c788-8be7-414d-9124-dfd1ffdc4934 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:34.958280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 157/318] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:50:34 2022] GET /identity/v3/role_assignments?user.id=2f3d14ad9e6c4fdda59307e744e32b02&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=cc9d3730bd5c4edcb1897fc5de4e031d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:34.963144 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66770db2-1e99-4e25-8140-38c4d972203c None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:34.963426 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66770db2-1e99-4e25-8140-38c4d972203c None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:34.963695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66770db2-1e99-4e25-8140-38c4d972203c None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:34.965425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 162/319] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:34.971136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72209c-1074-4691-95e6-a1710473912d None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:34.971496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72209c-1074-4691-95e6-a1710473912d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:34.971771 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72209c-1074-4691-95e6-a1710473912d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:34.992038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-ff72209c-1074-4691-95e6-a1710473912d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:34.993306 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ff72209c-1074-4691-95e6-a1710473912d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:35.060854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:35.068063 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 158/320] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:35.090375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66770db2-1e99-4e25-8140-38c4d972203c None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:35.094781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:35.225223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66770db2-1e99-4e25-8140-38c4d972203c None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:35.261244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f11fff-7a14-4343-83b5-638941a6f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:35.261641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f11fff-7a14-4343-83b5-638941a6f2ac None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:35.261914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f11fff-7a14-4343-83b5-638941a6f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:35.262185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f11fff-7a14-4343-83b5-638941a6f2ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:35.263320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f11fff-7a14-4343-83b5-638941a6f2ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:35.264828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f11fff-7a14-4343-83b5-638941a6f2ac None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:35.266649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 163/321] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:35.272083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72209c-1074-4691-95e6-a1710473912d None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:35.361764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557b41c-ad05-4517-a2ca-592133103e4e None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:35.362354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557b41c-ad05-4517-a2ca-592133103e4e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:35.362625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557b41c-ad05-4517-a2ca-592133103e4e None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:35.363976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 164/322] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:35.369149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77b967-ceaa-42c0-ad3e-7514952d8a16 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:35.369450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77b967-ceaa-42c0-ad3e-7514952d8a16 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:35.369695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77b967-ceaa-42c0-ad3e-7514952d8a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:35.376130 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72209c-1074-4691-95e6-a1710473912d None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:35.388312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-fc77b967-ceaa-42c0-ad3e-7514952d8a16 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:35.389709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fc77b967-ceaa-42c0-ad3e-7514952d8a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:35.414328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a4b4e-685f-43ef-923a-214b7e5f9729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:35.414955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a4b4e-685f-43ef-923a-214b7e5f9729 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:35.415238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a4b4e-685f-43ef-923a-214b7e5f9729 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:35.415503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a4b4e-685f-43ef-923a-214b7e5f9729 None admin] PATH_INFO: `/v3/groups` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:35.416342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a4b4e-685f-43ef-923a-214b7e5f9729 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:35.417641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a4b4e-685f-43ef-923a-214b7e5f9729 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:35.436160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 159/323] 10.222.0.49 () {60 vars in 1134 bytes} [Mon Aug 1 11:50:35 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:35.472356 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 165/324] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:35.494997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a4b4e-685f-43ef-923a-214b7e5f9729 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:35.551954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:35.625921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a4b4e-685f-43ef-923a-214b7e5f9729 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:35.662538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a6676-fe8f-44da-9c03-fc31e943140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:35.663237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a6676-fe8f-44da-9c03-fc31e943140f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:35.663543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a6676-fe8f-44da-9c03-fc31e943140f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:35.663817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a6676-fe8f-44da-9c03-fc31e943140f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:35.674730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a6676-fe8f-44da-9c03-fc31e943140f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-938a6676-fe8f-44da-9c03-fc31e943140f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:35.680432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:35.681356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:35.682158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 160/325] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:35.689338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77b967-ceaa-42c0-ad3e-7514952d8a16 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:35.691784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:35.745557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264f6ee-d0c8-4c5d-8665-91bb4437aafd None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:35.745838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264f6ee-d0c8-4c5d-8665-91bb4437aafd None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:35.746160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264f6ee-d0c8-4c5d-8665-91bb4437aafd None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:35.747715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 161/326] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:35.753308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303a58-65c0-4cd6-8450-20bf9262a2e1 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:35.753915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303a58-65c0-4cd6-8450-20bf9262a2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:35.754208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99303a58-65c0-4cd6-8450-20bf9262a2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:35.776009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-99303a58-65c0-4cd6-8450-20bf9262a2e1 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:35.777245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-99303a58-65c0-4cd6-8450-20bf9262a2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:35.811451 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77b967-ceaa-42c0-ad3e-7514952d8a16 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:35.863710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a08c3-9cd0-403c-9a9a-336941fe3526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:35.864656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a08c3-9cd0-403c-9a9a-336941fe3526 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:35.864935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a08c3-9cd0-403c-9a9a-336941fe3526 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:35.865204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a08c3-9cd0-403c-9a9a-336941fe3526 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:35.865960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a08c3-9cd0-403c-9a9a-336941fe3526 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:35.866259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a08c3-9cd0-403c-9a9a-336941fe3526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:35.867696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a08c3-9cd0-403c-9a9a-336941fe3526 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:35.868970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 162/327] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:35.877489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 166/328] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:35.883285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99303a58-65c0-4cd6-8450-20bf9262a2e1 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:35.902607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a08c3-9cd0-403c-9a9a-336941fe3526 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:35.928817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:35.989935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:36.010543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99303a58-65c0-4cd6-8450-20bf9262a2e1 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:36.048523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83a1a4-5c3e-4973-84d3-0c4aae9bf854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:36.049808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83a1a4-5c3e-4973-84d3-0c4aae9bf854 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:36.049808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83a1a4-5c3e-4973-84d3-0c4aae9bf854 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:36.049808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83a1a4-5c3e-4973-84d3-0c4aae9bf854 None admin] PATH_INFO: `/v3/users/49216cd01efe4743b774907c083ad0aa` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:36.051008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83a1a4-5c3e-4973-84d3-0c4aae9bf854 None admin] RBAC: Authorizing `identity:get_user(user_id=49216cd01efe4743b774907c083ad0aa)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:36.052516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83a1a4-5c3e-4973-84d3-0c4aae9bf854 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:36.054231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 163/329] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:35 2022] GET /identity/v3/users/49216cd01efe4743b774907c083ad0aa => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:36.056249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a08c3-9cd0-403c-9a9a-336941fe3526 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:36.060035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83a1a4-5c3e-4973-84d3-0c4aae9bf854 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:36.102549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1383d59-a3b0-44fc-af70-ba716ac13c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:36.104064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1383d59-a3b0-44fc-af70-ba716ac13c36 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:36.104623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1383d59-a3b0-44fc-af70-ba716ac13c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:36.105157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1383d59-a3b0-44fc-af70-ba716ac13c36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:36.123022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1383d59-a3b0-44fc-af70-ba716ac13c36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-e1383d59-a3b0-44fc-af70-ba716ac13c36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:36.130704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:36.131630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:36.132524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:36.140481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 167/330] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:36.142319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1383d59-a3b0-44fc-af70-ba716ac13c36 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:36.219731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83a1a4-5c3e-4973-84d3-0c4aae9bf854 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:36.222915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:36.263169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbcb6c-62a8-4850-9966-f74926cc8aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:36.264417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbcb6c-62a8-4850-9966-f74926cc8aeb None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:36.264950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbcb6c-62a8-4850-9966-f74926cc8aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:36.265451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fbcb6c-62a8-4850-9966-f74926cc8aeb None admin] PATH_INFO: `/v3/system/users/49216cd01efe4743b774907c083ad0aa/roles/cc9d3730bd5c4edcb1897fc5de4e031d` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:36.267240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbcb6c-62a8-4850-9966-f74926cc8aeb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=49216cd01efe4743b774907c083ad0aa, role_id=cc9d3730bd5c4edcb1897fc5de4e031d)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:36.268083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1383d59-a3b0-44fc-af70-ba716ac13c36 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:36.269194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fbcb6c-62a8-4850-9966-f74926cc8aeb None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:36.285774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 164/331] 10.222.0.49 () {58 vars in 1334 bytes} [Mon Aug 1 11:50:36 2022] PUT /identity/v3/system/users/49216cd01efe4743b774907c083ad0aa/roles/cc9d3730bd5c4edcb1897fc5de4e031d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:36.301786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:36.320334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a1186-e5e1-4190-9d4f-c6aee6b32723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:36.320981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a1186-e5e1-4190-9d4f-c6aee6b32723 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:36.321308 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a1186-e5e1-4190-9d4f-c6aee6b32723 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:36.321588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a1186-e5e1-4190-9d4f-c6aee6b32723 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:36.322356 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a1186-e5e1-4190-9d4f-c6aee6b32723 None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:36.322622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a1186-e5e1-4190-9d4f-c6aee6b32723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:36.324029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a1186-e5e1-4190-9d4f-c6aee6b32723 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:36.336556 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 168/332] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:36.342788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbcb6c-62a8-4850-9966-f74926cc8aeb None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:36.503946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fbcb6c-62a8-4850-9966-f74926cc8aeb None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:36.541439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67603fc5-f012-478a-81cb-7e62dd13ace8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:36.542413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67603fc5-f012-478a-81cb-7e62dd13ace8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:36.542892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67603fc5-f012-478a-81cb-7e62dd13ace8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:36.543338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67603fc5-f012-478a-81cb-7e62dd13ace8 None admin] PATH_INFO: `/v3/users/7f9bc4a3b3d84886b2586d45cb4e461b` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:36.544757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67603fc5-f012-478a-81cb-7e62dd13ace8 None admin] RBAC: Authorizing `identity:get_user(user_id=7f9bc4a3b3d84886b2586d45cb4e461b)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:36.546443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67603fc5-f012-478a-81cb-7e62dd13ace8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:36.550935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 165/333] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:36 2022] GET /identity/v3/users/7f9bc4a3b3d84886b2586d45cb4e461b => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:36.555069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a1186-e5e1-4190-9d4f-c6aee6b32723 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:36.667376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a1186-e5e1-4190-9d4f-c6aee6b32723 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:36.703079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70218bed-52d3-4491-8b9a-65848ed39676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:36.704471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218bed-52d3-4491-8b9a-65848ed39676 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:36.704952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218bed-52d3-4491-8b9a-65848ed39676 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:36.705395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218bed-52d3-4491-8b9a-65848ed39676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:36.706385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70218bed-52d3-4491-8b9a-65848ed39676 None admin] RBAC: Adding query filter params (role.id=4f5140de9df4440a8fb63d93249d14d1, user.id=7f9bc4a3b3d84886b2586d45cb4e461b, scope.system=all) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:36.706869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70218bed-52d3-4491-8b9a-65848ed39676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:36.708508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70218bed-52d3-4491-8b9a-65848ed39676 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:36.724506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 169/334] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Aug 1 11:50:36 2022] GET /identity/v3/role_assignments?user.id=7f9bc4a3b3d84886b2586d45cb4e461b&scope.system=all&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 510 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:37.945676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84cc1e-f648-41b6-b125-f74c76957cf1 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:37.946724 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84cc1e-f648-41b6-b125-f74c76957cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:37.947198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84cc1e-f648-41b6-b125-f74c76957cf1 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:37.949241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 166/335] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:37.954400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672ed3c-2331-4cd1-839f-07d6f04a00e2 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:37.955286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672ed3c-2331-4cd1-839f-07d6f04a00e2 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:37.955756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672ed3c-2331-4cd1-839f-07d6f04a00e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:37.976780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b672ed3c-2331-4cd1-839f-07d6f04a00e2 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:37.978363 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b672ed3c-2331-4cd1-839f-07d6f04a00e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:38.028304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:38.065095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 170/336] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:38.100132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84cc1e-f648-41b6-b125-f74c76957cf1 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:38.143031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:38.221077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84cc1e-f648-41b6-b125-f74c76957cf1 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:38.226075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7acc0-f2bc-431b-958f-eeadf4347c73 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:38.226364 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7acc0-f2bc-431b-958f-eeadf4347c73 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:38.226642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7acc0-f2bc-431b-958f-eeadf4347c73 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:38.228357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 171/337] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:38.249259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eeb58-6e06-4c45-a4e4-f03ef395fa1b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:38.249547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eeb58-6e06-4c45-a4e4-f03ef395fa1b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:38.249828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2eeb58-6e06-4c45-a4e4-f03ef395fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:38.278540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805a332-b233-485d-87e3-0900c149b9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:38.279084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805a332-b233-485d-87e3-0900c149b9cd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:38.279362 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805a332-b233-485d-87e3-0900c149b9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:38.279632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805a332-b233-485d-87e3-0900c149b9cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:38.289355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ea2eeb58-6e06-4c45-a4e4-f03ef395fa1b None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:38.290841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ea2eeb58-6e06-4c45-a4e4-f03ef395fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:38.301248 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805a332-b233-485d-87e3-0900c149b9cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-a805a332-b233-485d-87e3-0900c149b9cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:38.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:38.309934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:38.310882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 167/338] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:38.321619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805a332-b233-485d-87e3-0900c149b9cd None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:38.402605 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 172/339] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:38.440567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2eeb58-6e06-4c45-a4e4-f03ef395fa1b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:38.542041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:38.552630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805a332-b233-485d-87e3-0900c149b9cd None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:38.626596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8054bb6-5112-479a-95d6-2996ded6bf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:38.627262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8054bb6-5112-479a-95d6-2996ded6bf1e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:38.627591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8054bb6-5112-479a-95d6-2996ded6bf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:38.627864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8054bb6-5112-479a-95d6-2996ded6bf1e None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:38.628740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8054bb6-5112-479a-95d6-2996ded6bf1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:38.628956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8054bb6-5112-479a-95d6-2996ded6bf1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:38.630444 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8054bb6-5112-479a-95d6-2996ded6bf1e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:38.631494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2eeb58-6e06-4c45-a4e4-f03ef395fa1b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:38.635838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:38.648256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 168/340] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:38.653947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8054bb6-5112-479a-95d6-2996ded6bf1e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:38.694671 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4e09d-6393-4131-a337-e607534c82e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:38.695342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e09d-6393-4131-a337-e607534c82e6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:38.695624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e09d-6393-4131-a337-e607534c82e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:38.695900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4e09d-6393-4131-a337-e607534c82e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:38.717387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4e09d-6393-4131-a337-e607534c82e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-9fb4e09d-6393-4131-a337-e607534c82e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:38.723174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:38.724072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:38.724910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 173/341] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:38.733776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4e09d-6393-4131-a337-e607534c82e6 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:38.857504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8054bb6-5112-479a-95d6-2996ded6bf1e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:38.913213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a65aa6-72cc-42a5-a6bc-4c732d855d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:38.913785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a65aa6-72cc-42a5-a6bc-4c732d855d12 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:38.914068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a65aa6-72cc-42a5-a6bc-4c732d855d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:38.914341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a65aa6-72cc-42a5-a6bc-4c732d855d12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:38.956689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:38.984440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4e09d-6393-4131-a337-e607534c82e6 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:39.007888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a65aa6-72cc-42a5-a6bc-4c732d855d12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:39.009438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a65aa6-72cc-42a5-a6bc-4c732d855d12 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:39.056595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc73026-8a82-4598-9ffe-ee8119f3ceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:39.057258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc73026-8a82-4598-9ffe-ee8119f3ceac None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:39.057541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc73026-8a82-4598-9ffe-ee8119f3ceac None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:39.057816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc73026-8a82-4598-9ffe-ee8119f3ceac None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:39.058565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc73026-8a82-4598-9ffe-ee8119f3ceac None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:39.058876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc73026-8a82-4598-9ffe-ee8119f3ceac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:39.060276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc73026-8a82-4598-9ffe-ee8119f3ceac None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:39.072147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 174/342] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:39.073536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516bd8b-663c-47f2-9d67-f1092a134827 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:39.073785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516bd8b-663c-47f2-9d67-f1092a134827 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:39.074064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516bd8b-663c-47f2-9d67-f1092a134827 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:39.075317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 175/343] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:39.081279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516bd8b-663c-47f2-9d67-f1092a134827 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:39.122500 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:39.124027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 169/344] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:39.125570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dadebc-1ba6-4671-b22b-4321667d5bf8 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:39.125858 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dadebc-1ba6-4671-b22b-4321667d5bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:39.126133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dadebc-1ba6-4671-b22b-4321667d5bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:39.148467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-20dadebc-1ba6-4671-b22b-4321667d5bf8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:39.149914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-20dadebc-1ba6-4671-b22b-4321667d5bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:39.215296 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516bd8b-663c-47f2-9d67-f1092a134827 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:39.220703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:39.225304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 170/345] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:39.229145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dadebc-1ba6-4671-b22b-4321667d5bf8 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:39.265059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29680b7d-66e8-4eee-b68e-b235791a9ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:39.265704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29680b7d-66e8-4eee-b68e-b235791a9ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:39.265988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29680b7d-66e8-4eee-b68e-b235791a9ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:39.266258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29680b7d-66e8-4eee-b68e-b235791a9ee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:39.341561 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dadebc-1ba6-4671-b22b-4321667d5bf8 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:39.347006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29680b7d-66e8-4eee-b68e-b235791a9ee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:39.348572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29680b7d-66e8-4eee-b68e-b235791a9ee0 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:39.381103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:39.409020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d495bc5-a09f-41eb-adcc-edf9b270efcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:39.410117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d495bc5-a09f-41eb-adcc-edf9b270efcb None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:39.410584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d495bc5-a09f-41eb-adcc-edf9b270efcb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:39.411041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d495bc5-a09f-41eb-adcc-edf9b270efcb None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:39.412524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d495bc5-a09f-41eb-adcc-edf9b270efcb None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:39.414262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d495bc5-a09f-41eb-adcc-edf9b270efcb None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:39.416505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 171/346] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:39 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:39.417475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d495bc5-a09f-41eb-adcc-edf9b270efcb None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:39.466082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 176/347] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:39.466875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29680b7d-66e8-4eee-b68e-b235791a9ee0 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:39.548654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:39.586938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29680b7d-66e8-4eee-b68e-b235791a9ee0 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:39.617119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:39.631086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97747e-e761-45a2-a349-772f095575cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:39.631913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97747e-e761-45a2-a349-772f095575cd None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:39.632205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97747e-e761-45a2-a349-772f095575cd None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:39.632582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97747e-e761-45a2-a349-772f095575cd None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:39.636930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d495bc5-a09f-41eb-adcc-edf9b270efcb None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:39.641558 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97747e-e761-45a2-a349-772f095575cd None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:39.643070 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97747e-e761-45a2-a349-772f095575cd None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:39.645249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 177/348] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:39 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:39.646103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97747e-e761-45a2-a349-772f095575cd None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:39.700076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14e038-8f42-48c4-aae3-3d73aefde1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:39.700779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14e038-8f42-48c4-aae3-3d73aefde1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:39.701060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14e038-8f42-48c4-aae3-3d73aefde1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:39.701331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c14e038-8f42-48c4-aae3-3d73aefde1d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:39.723509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c14e038-8f42-48c4-aae3-3d73aefde1d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-4c14e038-8f42-48c4-aae3-3d73aefde1d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:39.729770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:39.730696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:39.731498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 172/349] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:39.738112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14e038-8f42-48c4-aae3-3d73aefde1d9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:39.841947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97747e-e761-45a2-a349-772f095575cd None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:39.891363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c14e038-8f42-48c4-aae3-3d73aefde1d9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:39.910202 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:39.930853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f7824-923f-42ae-b7b6-c598330f452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:39.931906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f7824-923f-42ae-b7b6-c598330f452f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:39.932010 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af0dc1-1384-4242-83a6-c655e3c97080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:39.932567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f7824-923f-42ae-b7b6-c598330f452f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:39.932936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af0dc1-1384-4242-83a6-c655e3c97080 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:39.933235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f7824-923f-42ae-b7b6-c598330f452f None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:39.933333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af0dc1-1384-4242-83a6-c655e3c97080 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:39.933668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af0dc1-1384-4242-83a6-c655e3c97080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:39.934763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af0dc1-1384-4242-83a6-c655e3c97080 None admin] RBAC: Adding query filter params (group.id=af2b9bb190684cea998daed0b59c0ea4, role.id=8e13cd96bb124d52b5c8883fe960424f, scope.project.id=183e3860f51e45b8828e2c44923ca1f1) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:39.935076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af0dc1-1384-4242-83a6-c655e3c97080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:39.935358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f7824-923f-42ae-b7b6-c598330f452f None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:39.937040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f7824-923f-42ae-b7b6-c598330f452f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:39.937219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af0dc1-1384-4242-83a6-c655e3c97080 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:39.939581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 178/350] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:39 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:39.940636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f7824-923f-42ae-b7b6-c598330f452f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:39.945338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 173/351] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Aug 1 11:50:39 2022] GET /identity/v3/role_assignments?group.id=af2b9bb190684cea998daed0b59c0ea4&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=8e13cd96bb124d52b5c8883fe960424f => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:39.946809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25faf8-abb3-4848-b523-b0400a564640 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:39.947323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25faf8-abb3-4848-b523-b0400a564640 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:39.947696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25faf8-abb3-4848-b523-b0400a564640 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:39.951189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 174/352] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:39.951189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25faf8-abb3-4848-b523-b0400a564640 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:40.056934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:40.064356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25faf8-abb3-4848-b523-b0400a564640 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:40.111125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf49037-8914-4682-9756-b3f592cf9917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:40.112182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49037-8914-4682-9756-b3f592cf9917 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:40.112510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49037-8914-4682-9756-b3f592cf9917 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:40.112783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf49037-8914-4682-9756-b3f592cf9917 None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:40.114397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf49037-8914-4682-9756-b3f592cf9917 None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:40.116159 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf49037-8914-4682-9756-b3f592cf9917 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:40.118561 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 175/353] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:39 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:40.120050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6afe9-e414-43c1-8873-9c7ae86718c4 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:40.120369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6afe9-e414-43c1-8873-9c7ae86718c4 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:40.120649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6afe9-e414-43c1-8873-9c7ae86718c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:40.139034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f7824-923f-42ae-b7b6-c598330f452f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:40.142976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-63b6afe9-e414-43c1-8873-9c7ae86718c4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:40.144313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-63b6afe9-e414-43c1-8873-9c7ae86718c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:40.184762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd453239-8779-40d6-ae78-8d6c7c8cc166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:40.185453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd453239-8779-40d6-ae78-8d6c7c8cc166 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:40.185915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd453239-8779-40d6-ae78-8d6c7c8cc166 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:40.186187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd453239-8779-40d6-ae78-8d6c7c8cc166 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:40.186943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd453239-8779-40d6-ae78-8d6c7c8cc166 None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:40.187255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd453239-8779-40d6-ae78-8d6c7c8cc166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:40.188732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd453239-8779-40d6-ae78-8d6c7c8cc166 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:40.199179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:40.199502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 179/354] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:40.200943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876587d1-0b93-45b8-8cd2-5a64b6f5edec None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:40.201236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876587d1-0b93-45b8-8cd2-5a64b6f5edec None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:40.201544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876587d1-0b93-45b8-8cd2-5a64b6f5edec None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:40.203187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 180/355] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:40.203985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876587d1-0b93-45b8-8cd2-5a64b6f5edec None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:40.226900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 176/356] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:40.229629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3a75d-d4dc-4dc9-b8f9-6d74a0d3cec8 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:40.229950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3a75d-d4dc-4dc9-b8f9-6d74a0d3cec8 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:40.230238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3a75d-d4dc-4dc9-b8f9-6d74a0d3cec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:40.252986 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d0e3a75d-d4dc-4dc9-b8f9-6d74a0d3cec8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:40.254369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d0e3a75d-d4dc-4dc9-b8f9-6d74a0d3cec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:40.344631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 177/357] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:40.345575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3a75d-d4dc-4dc9-b8f9-6d74a0d3cec8 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:40.347006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876587d1-0b93-45b8-8cd2-5a64b6f5edec None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:40.375201 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:40.395063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8c248-d3b5-4af3-9d9e-0dbf32db440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:40.395780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8c248-d3b5-4af3-9d9e-0dbf32db440b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:40.396068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8c248-d3b5-4af3-9d9e-0dbf32db440b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:40.396388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8c248-d3b5-4af3-9d9e-0dbf32db440b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:40.397230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8c248-d3b5-4af3-9d9e-0dbf32db440b None admin] RBAC: Adding query filter params (group.id=af2b9bb190684cea998daed0b59c0ea4, role.id=4f5140de9df4440a8fb63d93249d14d1, scope.project.id=183e3860f51e45b8828e2c44923ca1f1) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:40.398191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8c248-d3b5-4af3-9d9e-0dbf32db440b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:40.399700 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8c248-d3b5-4af3-9d9e-0dbf32db440b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:40.408476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 181/358] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Aug 1 11:50:40 2022] GET /identity/v3/role_assignments?group.id=af2b9bb190684cea998daed0b59c0ea4&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:40.409554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8c248-d3b5-4af3-9d9e-0dbf32db440b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:40.480233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3a75d-d4dc-4dc9-b8f9-6d74a0d3cec8 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:40.492576 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:40.524306 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a11c6bc-ae49-4637-9c37-946fa4ac0400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:40.525877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11c6bc-ae49-4637-9c37-946fa4ac0400 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:40.526163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11c6bc-ae49-4637-9c37-946fa4ac0400 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:40.526442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a11c6bc-ae49-4637-9c37-946fa4ac0400 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:40.558445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8c248-d3b5-4af3-9d9e-0dbf32db440b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:40.605459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46732c0b-3b57-446c-ba0f-cf0c4ddceec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:40.607135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46732c0b-3b57-446c-ba0f-cf0c4ddceec9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:40.607653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46732c0b-3b57-446c-ba0f-cf0c4ddceec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:40.608137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46732c0b-3b57-446c-ba0f-cf0c4ddceec9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:40.613900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11c6bc-ae49-4637-9c37-946fa4ac0400 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:40.615423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a11c6bc-ae49-4637-9c37-946fa4ac0400 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:40.623771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46732c0b-3b57-446c-ba0f-cf0c4ddceec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-46732c0b-3b57-446c-ba0f-cf0c4ddceec9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:40.629879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:40.630821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 01 11:50:40.631737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:40.634723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 182/359] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:40.636576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46732c0b-3b57-446c-ba0f-cf0c4ddceec9 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:40.708224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:40.769989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 178/360] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:40.776207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279af465-d418-4566-b0d6-7822e162aa34 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:40.777342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279af465-d418-4566-b0d6-7822e162aa34 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:40.777821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279af465-d418-4566-b0d6-7822e162aa34 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:40.779590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 179/361] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:40.780571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279af465-d418-4566-b0d6-7822e162aa34 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:40.781398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46732c0b-3b57-446c-ba0f-cf0c4ddceec9 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:40.793630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:40.825856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f1b56-ba5d-4e3a-939b-1b4259a53956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:40.826619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f1b56-ba5d-4e3a-939b-1b4259a53956 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:40.826912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f1b56-ba5d-4e3a-939b-1b4259a53956 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:40.827187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f1b56-ba5d-4e3a-939b-1b4259a53956 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:40.841873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f1b56-ba5d-4e3a-939b-1b4259a53956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-156f1b56-ba5d-4e3a-939b-1b4259a53956 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:40.849184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:40.850067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:40.850925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 183/362] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:40.851678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f1b56-ba5d-4e3a-939b-1b4259a53956 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:40.955215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279af465-d418-4566-b0d6-7822e162aa34 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:40.971375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f1b56-ba5d-4e3a-939b-1b4259a53956 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:40.974633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:41.006795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d0f74-21fe-409d-afa2-35f66508f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:41.008457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d0f74-21fe-409d-afa2-35f66508f28a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.009194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d0f74-21fe-409d-afa2-35f66508f28a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.009684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d0f74-21fe-409d-afa2-35f66508f28a None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.010721 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d0f74-21fe-409d-afa2-35f66508f28a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:41.011236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d0f74-21fe-409d-afa2-35f66508f28a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:41.012995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28d0f74-21fe-409d-afa2-35f66508f28a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:41.016835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11ec5f3-a005-43d0-a369-8725e2bdbe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:41.017508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ec5f3-a005-43d0-a369-8725e2bdbe19 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.017791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ec5f3-a005-43d0-a369-8725e2bdbe19 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.018065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11ec5f3-a005-43d0-a369-8725e2bdbe19 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.019381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ec5f3-a005-43d0-a369-8725e2bdbe19 None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:41.020900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11ec5f3-a005-43d0-a369-8725e2bdbe19 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:41.022908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 184/363] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:40 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:41.023956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078cfd-6ffc-461d-96e6-82ba0bf70e01 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.024244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078cfd-6ffc-461d-96e6-82ba0bf70e01 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.024583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66078cfd-6ffc-461d-96e6-82ba0bf70e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.025300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 180/364] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:41.026426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d0f74-21fe-409d-afa2-35f66508f28a None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:41.054716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-66078cfd-6ffc-461d-96e6-82ba0bf70e01 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:41.056348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-66078cfd-6ffc-461d-96e6-82ba0bf70e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:41.119949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:41.150147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 185/365] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:41.151872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66078cfd-6ffc-461d-96e6-82ba0bf70e01 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:41.159637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d0f74-21fe-409d-afa2-35f66508f28a None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:41.200278 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486140f8-1126-48f4-b147-4b33a1ab7e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:41.201287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486140f8-1126-48f4-b147-4b33a1ab7e9c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.201643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486140f8-1126-48f4-b147-4b33a1ab7e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.201991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486140f8-1126-48f4-b147-4b33a1ab7e9c None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.202796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486140f8-1126-48f4-b147-4b33a1ab7e9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:41.203176 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486140f8-1126-48f4-b147-4b33a1ab7e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:41.204737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486140f8-1126-48f4-b147-4b33a1ab7e9c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:41.222813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 181/366] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:41.223729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486140f8-1126-48f4-b147-4b33a1ab7e9c None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:41.281411 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66078cfd-6ffc-461d-96e6-82ba0bf70e01 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:41.283313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:41.320781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c682cb-cc15-46b3-bf44-cdb54539029b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:41.321749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c682cb-cc15-46b3-bf44-cdb54539029b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.322204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c682cb-cc15-46b3-bf44-cdb54539029b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.324915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c682cb-cc15-46b3-bf44-cdb54539029b None admin] PATH_INFO: `/v3/users/49216cd01efe4743b774907c083ad0aa` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.326438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c682cb-cc15-46b3-bf44-cdb54539029b None admin] RBAC: Authorizing `identity:get_user(user_id=49216cd01efe4743b774907c083ad0aa)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:41.328093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c682cb-cc15-46b3-bf44-cdb54539029b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:41.330268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 186/367] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:50:41 2022] GET /identity/v3/users/49216cd01efe4743b774907c083ad0aa => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:41.331327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c682cb-cc15-46b3-bf44-cdb54539029b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:41.332583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486140f8-1126-48f4-b147-4b33a1ab7e9c None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:41.367087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:41.373460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59757c57-057b-409b-ac64-de1a80674bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:41.374487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59757c57-057b-409b-ac64-de1a80674bef None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.374979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59757c57-057b-409b-ac64-de1a80674bef None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.375441 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59757c57-057b-409b-ac64-de1a80674bef None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.376823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59757c57-057b-409b-ac64-de1a80674bef None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:41.378352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59757c57-057b-409b-ac64-de1a80674bef None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:41.380148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 182/368] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:41 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:41.381089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59757c57-057b-409b-ac64-de1a80674bef None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:41.444505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c682cb-cc15-46b3-bf44-cdb54539029b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:41.481796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db5553-536d-4b1a-b33a-51dacf1cef09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:41.482795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db5553-536d-4b1a-b33a-51dacf1cef09 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.483268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db5553-536d-4b1a-b33a-51dacf1cef09 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.483731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db5553-536d-4b1a-b33a-51dacf1cef09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.495079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db5553-536d-4b1a-b33a-51dacf1cef09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-19db5553-536d-4b1a-b33a-51dacf1cef09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:41.500431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:41.501301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:41.502084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:41.502674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59757c57-057b-409b-ac64-de1a80674bef None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:41.505037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 187/369] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:50:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:41.506149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db5553-536d-4b1a-b33a-51dacf1cef09 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:41.548362 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e895ad-a7f0-40ee-8fc8-a9029e559ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:41.549067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e895ad-a7f0-40ee-8fc8-a9029e559ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.549351 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e895ad-a7f0-40ee-8fc8-a9029e559ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.549624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e895ad-a7f0-40ee-8fc8-a9029e559ab5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.555613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:41.627850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db5553-536d-4b1a-b33a-51dacf1cef09 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:41.632879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e895ad-a7f0-40ee-8fc8-a9029e559ab5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:41.634641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e895ad-a7f0-40ee-8fc8-a9029e559ab5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:41.652701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:41.667196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975e553-5ab1-4f21-8c80-9abb5cbe95d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:41.667874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975e553-5ab1-4f21-8c80-9abb5cbe95d1 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.668094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975e553-5ab1-4f21-8c80-9abb5cbe95d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.668408 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975e553-5ab1-4f21-8c80-9abb5cbe95d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.669200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975e553-5ab1-4f21-8c80-9abb5cbe95d1 None admin] RBAC: Adding query filter params (role.id=cc9d3730bd5c4edcb1897fc5de4e031d, user.id=49216cd01efe4743b774907c083ad0aa, scope.system=all) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:41.669507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975e553-5ab1-4f21-8c80-9abb5cbe95d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:41.671145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975e553-5ab1-4f21-8c80-9abb5cbe95d1 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:41.686095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 188/370] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Aug 1 11:50:41 2022] GET /identity/v3/role_assignments?user.id=49216cd01efe4743b774907c083ad0aa&scope.system=all&role.id=cc9d3730bd5c4edcb1897fc5de4e031d => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:41.686391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975e553-5ab1-4f21-8c80-9abb5cbe95d1 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:41.758260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 183/371] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:41.759623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e895ad-a7f0-40ee-8fc8-a9029e559ab5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:41.790551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:41.807347 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975e553-5ab1-4f21-8c80-9abb5cbe95d1 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:41.858511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631526e-5630-496d-acb5-8250150fd8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:41.859162 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631526e-5630-496d-acb5-8250150fd8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.859447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631526e-5630-496d-acb5-8250150fd8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.859721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631526e-5630-496d-acb5-8250150fd8d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.860552 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631526e-5630-496d-acb5-8250150fd8d4 None admin] RBAC: Adding query filter params (group.id=af2b9bb190684cea998daed0b59c0ea4, role.id=4f5140de9df4440a8fb63d93249d14d1, scope.project.id=876fcd5989ee455a89866281d214740e) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:41.860863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631526e-5630-496d-acb5-8250150fd8d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:41.862797 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631526e-5630-496d-acb5-8250150fd8d4 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:41.869366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 189/372] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Aug 1 11:50:41 2022] GET /identity/v3/role_assignments?group.id=af2b9bb190684cea998daed0b59c0ea4&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:41.870254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631526e-5630-496d-acb5-8250150fd8d4 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:41.885140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e895ad-a7f0-40ee-8fc8-a9029e559ab5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:41.889015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:41.933648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef30d94-03a4-4319-a139-c92fc4feabd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:41.934745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef30d94-03a4-4319-a139-c92fc4feabd5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:41.935224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef30d94-03a4-4319-a139-c92fc4feabd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:41.935695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef30d94-03a4-4319-a139-c92fc4feabd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:41.936769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef30d94-03a4-4319-a139-c92fc4feabd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:41.937289 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef30d94-03a4-4319-a139-c92fc4feabd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:41.938960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef30d94-03a4-4319-a139-c92fc4feabd5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:41.949975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 184/373] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:41.955806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef30d94-03a4-4319-a139-c92fc4feabd5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:42.097221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631526e-5630-496d-acb5-8250150fd8d4 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:42.159665 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f6b0f-f7a1-463c-82fc-98f10239a06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:42.160412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f6b0f-f7a1-463c-82fc-98f10239a06b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:42.162372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f6b0f-f7a1-463c-82fc-98f10239a06b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:42.162372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f6b0f-f7a1-463c-82fc-98f10239a06b None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:42.163670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f6b0f-f7a1-463c-82fc-98f10239a06b None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:42.165199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f6b0f-f7a1-463c-82fc-98f10239a06b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:42.166958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 190/374] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:41 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:42.173449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f6b0f-f7a1-463c-82fc-98f10239a06b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:42.199544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:42.217652 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef30d94-03a4-4319-a139-c92fc4feabd5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:42.284882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78d6ff-98cc-494d-84ad-1f47f2b80fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:42.285518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78d6ff-98cc-494d-84ad-1f47f2b80fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:42.285798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78d6ff-98cc-494d-84ad-1f47f2b80fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:42.286743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78d6ff-98cc-494d-84ad-1f47f2b80fd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:42.297792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f6b0f-f7a1-463c-82fc-98f10239a06b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:42.335066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f07f3-2e71-42e6-8407-7e8bc34fdb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:42.336064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f07f3-2e71-42e6-8407-7e8bc34fdb54 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:42.336527 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f07f3-2e71-42e6-8407-7e8bc34fdb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:42.336974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f07f3-2e71-42e6-8407-7e8bc34fdb54 None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:42.338445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f07f3-2e71-42e6-8407-7e8bc34fdb54 None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:42.340138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94f07f3-2e71-42e6-8407-7e8bc34fdb54 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:42.342495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 191/375] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:42 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:42.347943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:42.349490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f07f3-2e71-42e6-8407-7e8bc34fdb54 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:42.397504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78d6ff-98cc-494d-84ad-1f47f2b80fd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:42.398738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd78d6ff-98cc-494d-84ad-1f47f2b80fd5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:42.450330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:42.468366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f07f3-2e71-42e6-8407-7e8bc34fdb54 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:42.520776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd5842-01c9-49ff-9cb1-88ae99f9dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:42.521533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd5842-01c9-49ff-9cb1-88ae99f9dd66 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:42.521878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd5842-01c9-49ff-9cb1-88ae99f9dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:42.522266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd5842-01c9-49ff-9cb1-88ae99f9dd66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:42.523138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd5842-01c9-49ff-9cb1-88ae99f9dd66 None admin] RBAC: Adding query filter params (group.id=af2b9bb190684cea998daed0b59c0ea4, role.id=8e13cd96bb124d52b5c8883fe960424f, scope.project.id=876fcd5989ee455a89866281d214740e) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:42.523517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd5842-01c9-49ff-9cb1-88ae99f9dd66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:42.525192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd5842-01c9-49ff-9cb1-88ae99f9dd66 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:42.527864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 185/376] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:42.535883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 192/377] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Aug 1 11:50:42 2022] GET /identity/v3/role_assignments?group.id=af2b9bb190684cea998daed0b59c0ea4&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=8e13cd96bb124d52b5c8883fe960424f => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:42.537200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78d6ff-98cc-494d-84ad-1f47f2b80fd5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:42.575862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd997-1fae-4d6a-b31c-6822594d61ee None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:42.576329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd997-1fae-4d6a-b31c-6822594d61ee None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:42.576708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2dd997-1fae-4d6a-b31c-6822594d61ee None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:42.578664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 193/378] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:42.584448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a11543-1dc5-46ad-88ff-e67db3d722d8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:42.584897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a11543-1dc5-46ad-88ff-e67db3d722d8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:42.585295 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a11543-1dc5-46ad-88ff-e67db3d722d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:42.606624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e7a11543-1dc5-46ad-88ff-e67db3d722d8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:42.607986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e7a11543-1dc5-46ad-88ff-e67db3d722d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:42.612841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:42.660359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78d6ff-98cc-494d-84ad-1f47f2b80fd5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:42.719663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 194/379] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:42.724373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abba561-ec5b-4284-a70d-d18e7b3630ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:42.724373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abba561-ec5b-4284-a70d-d18e7b3630ee None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:42.724373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abba561-ec5b-4284-a70d-d18e7b3630ee None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:42.728712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abba561-ec5b-4284-a70d-d18e7b3630ee None admin] PATH_INFO: `/v3/projects/a841fcca183a4d31ac0320c337760588` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:42.730288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abba561-ec5b-4284-a70d-d18e7b3630ee None admin] RBAC: Authorizing `identity:get_project(project_id=a841fcca183a4d31ac0320c337760588)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:42.731827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abba561-ec5b-4284-a70d-d18e7b3630ee None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:42.733814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 186/380] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:42 2022] GET /identity/v3/projects/a841fcca183a4d31ac0320c337760588 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:42.740483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a11543-1dc5-46ad-88ff-e67db3d722d8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:42.768099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abba561-ec5b-4284-a70d-d18e7b3630ee None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:42.860062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:42.877179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a11543-1dc5-46ad-88ff-e67db3d722d8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:42.921702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43198bb6-ba92-487d-8125-bb2de0a6bb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:42.923010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43198bb6-ba92-487d-8125-bb2de0a6bb11 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:42.923494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43198bb6-ba92-487d-8125-bb2de0a6bb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:42.923957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43198bb6-ba92-487d-8125-bb2de0a6bb11 None admin] PATH_INFO: `/v3/groups/a618bb16314a48b6aba0809731c489d6` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:42.931650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43198bb6-ba92-487d-8125-bb2de0a6bb11 None admin] RBAC: Authorizing `identity:get_group(group_id=a618bb16314a48b6aba0809731c489d6)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:42.933682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43198bb6-ba92-487d-8125-bb2de0a6bb11 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:42.935610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 195/381] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:42 2022] GET /identity/v3/groups/a618bb16314a48b6aba0809731c489d6 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:42.942819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43198bb6-ba92-487d-8125-bb2de0a6bb11 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:43.019909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abba561-ec5b-4284-a70d-d18e7b3630ee None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:43.062475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:43.069933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1031c-fb03-470b-90da-dba3f0f680c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:43.071138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1031c-fb03-470b-90da-dba3f0f680c9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:43.071516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1031c-fb03-470b-90da-dba3f0f680c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:43.071892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1031c-fb03-470b-90da-dba3f0f680c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:43.092686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee1031c-fb03-470b-90da-dba3f0f680c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:43.095634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43198bb6-ba92-487d-8125-bb2de0a6bb11 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-6ee1031c-fb03-470b-90da-dba3f0f680c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:43.099517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:43.100763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:43.101552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:43.108656 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 187/382] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:43.109957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1031c-fb03-470b-90da-dba3f0f680c9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:43.141148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b46935-03a2-411b-92ff-f7fde9c30b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:43.141763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b46935-03a2-411b-92ff-f7fde9c30b4e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:43.142047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b46935-03a2-411b-92ff-f7fde9c30b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:43.142374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b46935-03a2-411b-92ff-f7fde9c30b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:43.143592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b46935-03a2-411b-92ff-f7fde9c30b4e None admin] RBAC: Adding query filter params (group.id=a618bb16314a48b6aba0809731c489d6, role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=a841fcca183a4d31ac0320c337760588) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:43.143938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b46935-03a2-411b-92ff-f7fde9c30b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:43.145811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b46935-03a2-411b-92ff-f7fde9c30b4e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:43.156339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 196/383] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Aug 1 11:50:42 2022] GET /identity/v3/role_assignments?group.id=a618bb16314a48b6aba0809731c489d6&scope.project.id=a841fcca183a4d31ac0320c337760588&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:43.233415 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee1031c-fb03-470b-90da-dba3f0f680c9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:43.275939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b4417-7f69-4665-b594-4b83ea3b1043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:43.276627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b4417-7f69-4665-b594-4b83ea3b1043 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:43.276910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b4417-7f69-4665-b594-4b83ea3b1043 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:43.277186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b4417-7f69-4665-b594-4b83ea3b1043 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:43.277930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b4417-7f69-4665-b594-4b83ea3b1043 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:43.278238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b4417-7f69-4665-b594-4b83ea3b1043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:43.279659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b4417-7f69-4665-b594-4b83ea3b1043 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:43.296794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 188/384] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:43.305552 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b46935-03a2-411b-92ff-f7fde9c30b4e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:43.327976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:43.413939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6527e6-4018-4ec8-aea6-1fedb5eb0ab5 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:43.414116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6527e6-4018-4ec8-aea6-1fedb5eb0ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:43.414386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6527e6-4018-4ec8-aea6-1fedb5eb0ab5 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:43.415964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 189/385] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:43.424928 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a178a84-4aa8-499b-8616-9e693236f068 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:43.425243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a178a84-4aa8-499b-8616-9e693236f068 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:43.425485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a178a84-4aa8-499b-8616-9e693236f068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:43.453491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9a178a84-4aa8-499b-8616-9e693236f068 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:43.454798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9a178a84-4aa8-499b-8616-9e693236f068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:43.487721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b46935-03a2-411b-92ff-f7fde9c30b4e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:43.560883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:43.561879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f72246-f405-4aef-bca3-54f949fd150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:43.562753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f72246-f405-4aef-bca3-54f949fd150f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:43.563196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f72246-f405-4aef-bca3-54f949fd150f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:43.563619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f72246-f405-4aef-bca3-54f949fd150f None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:43.565012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f72246-f405-4aef-bca3-54f949fd150f None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:43.566741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f72246-f405-4aef-bca3-54f949fd150f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:43.569002 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 197/386] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:43 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:43.573417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 190/387] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:43.574534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f72246-f405-4aef-bca3-54f949fd150f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:43.624497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a178a84-4aa8-499b-8616-9e693236f068 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:43.680471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:43.707548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f72246-f405-4aef-bca3-54f949fd150f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:43.768156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a178a84-4aa8-499b-8616-9e693236f068 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:43.771714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9d6c4-24c4-450f-abf7-8b48698a8837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:43.776163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9d6c4-24c4-450f-abf7-8b48698a8837 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:43.776687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9d6c4-24c4-450f-abf7-8b48698a8837 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:43.777125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9d6c4-24c4-450f-abf7-8b48698a8837 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:43.778941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9d6c4-24c4-450f-abf7-8b48698a8837 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:43.785230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9d6c4-24c4-450f-abf7-8b48698a8837 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:43.790497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 198/388] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:43 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:43.793722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9d6c4-24c4-450f-abf7-8b48698a8837 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:43.834796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294192ee-ace0-41ca-abb4-8825894fbd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:43.835468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294192ee-ace0-41ca-abb4-8825894fbd64 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:43.835748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294192ee-ace0-41ca-abb4-8825894fbd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:43.836021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294192ee-ace0-41ca-abb4-8825894fbd64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:43.860419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294192ee-ace0-41ca-abb4-8825894fbd64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-294192ee-ace0-41ca-abb4-8825894fbd64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:43.865944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:43.866899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:43.867740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 191/389] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:43.874231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294192ee-ace0-41ca-abb4-8825894fbd64 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:43.948785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:43.984004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9d6c4-24c4-450f-abf7-8b48698a8837 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:44.053134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e8659-e43d-46ad-8827-3845fb3b6af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:44.054386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e8659-e43d-46ad-8827-3845fb3b6af3 None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:44.056592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294192ee-ace0-41ca-abb4-8825894fbd64 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:44.057095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e8659-e43d-46ad-8827-3845fb3b6af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:44.057609 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e8659-e43d-46ad-8827-3845fb3b6af3 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1/groups/af2b9bb190684cea998daed0b59c0ea4/roles/8e13cd96bb124d52b5c8883fe960424f` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:44.062042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e8659-e43d-46ad-8827-3845fb3b6af3 None admin] RBAC: Authorizing `identity:create_grant(project_id=183e3860f51e45b8828e2c44923ca1f1, group_id=af2b9bb190684cea998daed0b59c0ea4, role_id=8e13cd96bb124d52b5c8883fe960424f)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:44.063862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e8659-e43d-46ad-8827-3845fb3b6af3 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:44.096268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 199/390] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Aug 1 11:50:43 2022] PUT /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1/groups/af2b9bb190684cea998daed0b59c0ea4/roles/8e13cd96bb124d52b5c8883fe960424f => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 11:50:44.126850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79113f61-188c-4f02-83e7-48ba37d273b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:44.127497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79113f61-188c-4f02-83e7-48ba37d273b2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:44.127775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79113f61-188c-4f02-83e7-48ba37d273b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:44.128041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79113f61-188c-4f02-83e7-48ba37d273b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:44.128870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79113f61-188c-4f02-83e7-48ba37d273b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:44.129997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79113f61-188c-4f02-83e7-48ba37d273b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:44.131587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79113f61-188c-4f02-83e7-48ba37d273b2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:44.149359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 192/391] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:44.155151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e8659-e43d-46ad-8827-3845fb3b6af3 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:44.177787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:44.321566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e8659-e43d-46ad-8827-3845fb3b6af3 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:44.374319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb32be6-f19e-49f2-a94d-5a00b220323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:44.375101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb32be6-f19e-49f2-a94d-5a00b220323f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:44.375459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb32be6-f19e-49f2-a94d-5a00b220323f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:44.375737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb32be6-f19e-49f2-a94d-5a00b220323f None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:44.378829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb32be6-f19e-49f2-a94d-5a00b220323f None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:44.380392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb32be6-f19e-49f2-a94d-5a00b220323f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:44.382945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 200/392] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:44 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:44.388853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79113f61-188c-4f02-83e7-48ba37d273b2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:44.504486 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79113f61-188c-4f02-83e7-48ba37d273b2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:44.507092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:44.542124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb46bd-ae80-46ed-9fb6-da4f878ba672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:44.542793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb46bd-ae80-46ed-9fb6-da4f878ba672 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:44.543070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb46bd-ae80-46ed-9fb6-da4f878ba672 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:44.543343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb46bd-ae80-46ed-9fb6-da4f878ba672 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:44.544644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb46bd-ae80-46ed-9fb6-da4f878ba672 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:44.546102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb46bd-ae80-46ed-9fb6-da4f878ba672 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:44.548101 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 193/393] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:44 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:44.553793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb32be6-f19e-49f2-a94d-5a00b220323f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:44.651528 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:44.691296 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb32be6-f19e-49f2-a94d-5a00b220323f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:44.734333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad9cb4-ce70-41fc-9d1d-907c311e1021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:44.735333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad9cb4-ce70-41fc-9d1d-907c311e1021 None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:44.735760 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad9cb4-ce70-41fc-9d1d-907c311e1021 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:44.736038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad9cb4-ce70-41fc-9d1d-907c311e1021 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1/groups/af2b9bb190684cea998daed0b59c0ea4/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:44.738538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad9cb4-ce70-41fc-9d1d-907c311e1021 None admin] RBAC: Authorizing `identity:create_grant(project_id=183e3860f51e45b8828e2c44923ca1f1, group_id=af2b9bb190684cea998daed0b59c0ea4, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:44.740017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad9cb4-ce70-41fc-9d1d-907c311e1021 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:44.745822 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d8423-5ff9-4265-ad0b-17dfbb64de91 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:44.746164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d8423-5ff9-4265-ad0b-17dfbb64de91 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:44.746363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d8423-5ff9-4265-ad0b-17dfbb64de91 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:44.747993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 194/394] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:44.753844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb79640-1778-40bd-a811-3d5873f18c0b None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:44.754130 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb79640-1778-40bd-a811-3d5873f18c0b None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:44.754456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb79640-1778-40bd-a811-3d5873f18c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:44.775185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 201/395] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Aug 1 11:50:44 2022] PUT /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1/groups/af2b9bb190684cea998daed0b59c0ea4/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:44.782600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6eb79640-1778-40bd-a811-3d5873f18c0b None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:44.783967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6eb79640-1778-40bd-a811-3d5873f18c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:44.882607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 195/396] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:44.913770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad9cb4-ce70-41fc-9d1d-907c311e1021 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:45.039408 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad9cb4-ce70-41fc-9d1d-907c311e1021 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:45.081406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43229d30-f341-4793-ab59-fe155ebd38ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:45.082067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43229d30-f341-4793-ab59-fe155ebd38ee None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:45.082351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43229d30-f341-4793-ab59-fe155ebd38ee None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:45.082622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43229d30-f341-4793-ab59-fe155ebd38ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:45.086883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:45.100836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43229d30-f341-4793-ab59-fe155ebd38ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-43229d30-f341-4793-ab59-fe155ebd38ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:45.108056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:45.109009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:45.109774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 202/397] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:45.117302 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb79640-1778-40bd-a811-3d5873f18c0b None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:45.251030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb79640-1778-40bd-a811-3d5873f18c0b None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:45.252699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:45.299579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63753e2-e847-4340-9892-451bfb248d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:45.300245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63753e2-e847-4340-9892-451bfb248d6b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:45.300557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63753e2-e847-4340-9892-451bfb248d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:45.300841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63753e2-e847-4340-9892-451bfb248d6b None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:45.301600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63753e2-e847-4340-9892-451bfb248d6b None admin] RBAC: Adding query filter params (name=member) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:45.301905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63753e2-e847-4340-9892-451bfb248d6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:45.303415 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63753e2-e847-4340-9892-451bfb248d6b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:45.314897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 196/398] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:45.321427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43229d30-f341-4793-ab59-fe155ebd38ee None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:45.456478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43229d30-f341-4793-ab59-fe155ebd38ee None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:45.499091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ee0dc-1072-4989-9a0a-5e68de98ee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:45.499880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ee0dc-1072-4989-9a0a-5e68de98ee52 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:45.500157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ee0dc-1072-4989-9a0a-5e68de98ee52 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:45.500460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ee0dc-1072-4989-9a0a-5e68de98ee52 None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:45.501837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ee0dc-1072-4989-9a0a-5e68de98ee52 None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:45.503310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ee0dc-1072-4989-9a0a-5e68de98ee52 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:45.505338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 203/399] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:45 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:45.511129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63753e2-e847-4340-9892-451bfb248d6b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:45.633744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63753e2-e847-4340-9892-451bfb248d6b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:45.680064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcbf0a-1eaf-4f78-8c39-5652f3b84ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:45.680844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcbf0a-1eaf-4f78-8c39-5652f3b84ced None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:45.681125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcbf0a-1eaf-4f78-8c39-5652f3b84ced None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:45.681397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcbf0a-1eaf-4f78-8c39-5652f3b84ced None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:45.682604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcbf0a-1eaf-4f78-8c39-5652f3b84ced None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:45.684059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcbf0a-1eaf-4f78-8c39-5652f3b84ced None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:45.686064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 197/400] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:45 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:45.691904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ee0dc-1072-4989-9a0a-5e68de98ee52 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:45.731555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f49ce-9919-4301-aa3b-a4512d20fc5e None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:45.731887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f49ce-9919-4301-aa3b-a4512d20fc5e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:45.732122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f49ce-9919-4301-aa3b-a4512d20fc5e None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:45.733862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 198/401] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:45.739890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea63a7-ede4-42a4-9deb-e12de2dba6a5 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:45.740202 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea63a7-ede4-42a4-9deb-e12de2dba6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:45.740476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea63a7-ede4-42a4-9deb-e12de2dba6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:45.765726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8cea63a7-ede4-42a4-9deb-e12de2dba6a5 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:45.767491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8cea63a7-ede4-42a4-9deb-e12de2dba6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:45.783620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:45.830985 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:45.871882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ee0dc-1072-4989-9a0a-5e68de98ee52 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:45.911328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 199/402] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:45.933977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f2643-3ccd-46c0-ae8d-b0f6e063ea32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:45.934682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f2643-3ccd-46c0-ae8d-b0f6e063ea32 None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:45.934959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f2643-3ccd-46c0-ae8d-b0f6e063ea32 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:45.935229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f2643-3ccd-46c0-ae8d-b0f6e063ea32 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e/groups/af2b9bb190684cea998daed0b59c0ea4/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:45.938247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea63a7-ede4-42a4-9deb-e12de2dba6a5 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:45.949094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f2643-3ccd-46c0-ae8d-b0f6e063ea32 None admin] RBAC: Authorizing `identity:create_grant(project_id=876fcd5989ee455a89866281d214740e, group_id=af2b9bb190684cea998daed0b59c0ea4, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:45.950686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f2643-3ccd-46c0-ae8d-b0f6e063ea32 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:45.983827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 204/403] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Aug 1 11:50:45 2022] PUT /identity/v3/projects/876fcd5989ee455a89866281d214740e/groups/af2b9bb190684cea998daed0b59c0ea4/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:46.073749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea63a7-ede4-42a4-9deb-e12de2dba6a5 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:46.114381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae6a62-158d-402d-bb00-4910cdcdfd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:46.115060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae6a62-158d-402d-bb00-4910cdcdfd11 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:46.115336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae6a62-158d-402d-bb00-4910cdcdfd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:46.115692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae6a62-158d-402d-bb00-4910cdcdfd11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:46.134789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae6a62-158d-402d-bb00-4910cdcdfd11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-3eae6a62-158d-402d-bb00-4910cdcdfd11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:46.139923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:46.140918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:46.141688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 200/404] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:46.147917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f2643-3ccd-46c0-ae8d-b0f6e063ea32 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:46.169269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a0509-95f3-4e4c-9eb6-fc7808276a7d None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:46.169602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a0509-95f3-4e4c-9eb6-fc7808276a7d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:46.169961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a0509-95f3-4e4c-9eb6-fc7808276a7d None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:46.171698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 201/405] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:46.182247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86841d9a-65d5-4bde-9701-68e5298de9a3 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:46.182574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86841d9a-65d5-4bde-9701-68e5298de9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:46.183376 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86841d9a-65d5-4bde-9701-68e5298de9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:46.212866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-86841d9a-65d5-4bde-9701-68e5298de9a3 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:46.212866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-86841d9a-65d5-4bde-9701-68e5298de9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:46.248977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:46.313696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 202/406] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:46.325413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f2643-3ccd-46c0-ae8d-b0f6e063ea32 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:46.339237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86841d9a-65d5-4bde-9701-68e5298de9a3 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:46.353289 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:46.370272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136a1c0-f18d-45b2-a668-f11d033c258a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:46.371343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136a1c0-f18d-45b2-a668-f11d033c258a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:46.371343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136a1c0-f18d-45b2-a668-f11d033c258a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:46.371774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136a1c0-f18d-45b2-a668-f11d033c258a None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:46.372648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136a1c0-f18d-45b2-a668-f11d033c258a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:46.373064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136a1c0-f18d-45b2-a668-f11d033c258a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:46.374581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136a1c0-f18d-45b2-a668-f11d033c258a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:46.393786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 205/407] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:46.414862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136a1c0-f18d-45b2-a668-f11d033c258a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:46.469208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86841d9a-65d5-4bde-9701-68e5298de9a3 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:46.512241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bffa85-d8f0-439b-96f0-f22d5960d954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:46.513633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bffa85-d8f0-439b-96f0-f22d5960d954 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:46.514105 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bffa85-d8f0-439b-96f0-f22d5960d954 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:46.514513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bffa85-d8f0-439b-96f0-f22d5960d954 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:46.531739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bffa85-d8f0-439b-96f0-f22d5960d954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-39bffa85-d8f0-439b-96f0-f22d5960d954 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:46.541929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:46.542926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:46.543703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:46.546044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 203/408] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:50:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:46.556457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bffa85-d8f0-439b-96f0-f22d5960d954 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:46.563344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136a1c0-f18d-45b2-a668-f11d033c258a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:46.589276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:46.641506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75066f2-d11d-4236-81fb-306fc8e9077a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:46.642418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75066f2-d11d-4236-81fb-306fc8e9077a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:46.642837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75066f2-d11d-4236-81fb-306fc8e9077a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:46.643243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75066f2-d11d-4236-81fb-306fc8e9077a None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:46.644865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75066f2-d11d-4236-81fb-306fc8e9077a None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:46.646512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75066f2-d11d-4236-81fb-306fc8e9077a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:46.651523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 206/409] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:46 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:46.658293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75066f2-d11d-4236-81fb-306fc8e9077a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:46.767380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bffa85-d8f0-439b-96f0-f22d5960d954 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:46.768687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:46.843159 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733a672-41ca-4ece-a241-bbbd77432d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:46.843808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733a672-41ca-4ece-a241-bbbd77432d1b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:46.844636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733a672-41ca-4ece-a241-bbbd77432d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:46.844916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733a672-41ca-4ece-a241-bbbd77432d1b None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:46.845775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733a672-41ca-4ece-a241-bbbd77432d1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:46.846161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733a672-41ca-4ece-a241-bbbd77432d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:46.847567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733a672-41ca-4ece-a241-bbbd77432d1b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:46.872708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 204/410] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:46.873922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733a672-41ca-4ece-a241-bbbd77432d1b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:46.876167 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75066f2-d11d-4236-81fb-306fc8e9077a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:46.951518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd7dc9-9504-42eb-866d-43e5d25af4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:46.952180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd7dc9-9504-42eb-866d-43e5d25af4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:46.952482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd7dc9-9504-42eb-866d-43e5d25af4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:46.952750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd7dc9-9504-42eb-866d-43e5d25af4d5 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:46.954625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd7dc9-9504-42eb-866d-43e5d25af4d5 None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:46.956081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd7dc9-9504-42eb-866d-43e5d25af4d5 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:46.959004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 207/411] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:46 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:46.974758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd7dc9-9504-42eb-866d-43e5d25af4d5 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:47.014596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:47.042093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733a672-41ca-4ece-a241-bbbd77432d1b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:47.079353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0635d-cc18-498c-903e-32e5727e0387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:47.080012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0635d-cc18-498c-903e-32e5727e0387 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:47.080303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0635d-cc18-498c-903e-32e5727e0387 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:47.080602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0635d-cc18-498c-903e-32e5727e0387 None admin] PATH_INFO: `/v3/groups/a618bb16314a48b6aba0809731c489d6` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:47.081781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0635d-cc18-498c-903e-32e5727e0387 None admin] RBAC: Authorizing `identity:get_group(group_id=a618bb16314a48b6aba0809731c489d6)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:47.083346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0635d-cc18-498c-903e-32e5727e0387 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:47.085231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 205/412] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:46 2022] GET /identity/v3/groups/a618bb16314a48b6aba0809731c489d6 => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:47.086477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d52cc-af63-4b46-837e-5651eeb669ec None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:47.086757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d52cc-af63-4b46-837e-5651eeb669ec None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:47.087027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d52cc-af63-4b46-837e-5651eeb669ec None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:47.096759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 206/413] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:47.096759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d52cc-af63-4b46-837e-5651eeb669ec None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:47.103512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd7dc9-9504-42eb-866d-43e5d25af4d5 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:47.130557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:47.149791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5167b5-7937-46ce-9843-ce969a6c14e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:47.150476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5167b5-7937-46ce-9843-ce969a6c14e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:47.150759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5167b5-7937-46ce-9843-ce969a6c14e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:47.151149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5167b5-7937-46ce-9843-ce969a6c14e0 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e/groups/af2b9bb190684cea998daed0b59c0ea4/roles/8e13cd96bb124d52b5c8883fe960424f` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:47.154682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5167b5-7937-46ce-9843-ce969a6c14e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=876fcd5989ee455a89866281d214740e, group_id=af2b9bb190684cea998daed0b59c0ea4, role_id=8e13cd96bb124d52b5c8883fe960424f)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:47.156492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5167b5-7937-46ce-9843-ce969a6c14e0 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:47.184688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 208/414] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Aug 1 11:50:46 2022] PUT /identity/v3/projects/876fcd5989ee455a89866281d214740e/groups/af2b9bb190684cea998daed0b59c0ea4/roles/8e13cd96bb124d52b5c8883fe960424f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 11:50:47.186247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c2af-3239-4a20-b50a-fdcdab4a976a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:47.186589 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c2af-3239-4a20-b50a-fdcdab4a976a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:47.186872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695c2af-3239-4a20-b50a-fdcdab4a976a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:47.217506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a695c2af-3239-4a20-b50a-fdcdab4a976a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:47.218907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a695c2af-3239-4a20-b50a-fdcdab4a976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:47.248284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d52cc-af63-4b46-837e-5651eeb669ec None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:47.295293 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf59aad-06e8-47d9-90af-3c73e424f428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:47.296441 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf59aad-06e8-47d9-90af-3c73e424f428 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:47.296952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf59aad-06e8-47d9-90af-3c73e424f428 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:47.297807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf59aad-06e8-47d9-90af-3c73e424f428 None admin] PATH_INFO: `/v3/projects/a841fcca183a4d31ac0320c337760588` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:47.299458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf59aad-06e8-47d9-90af-3c73e424f428 None admin] RBAC: Authorizing `identity:get_project(project_id=a841fcca183a4d31ac0320c337760588)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:47.301768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf59aad-06e8-47d9-90af-3c73e424f428 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:47.303932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 207/415] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:47 2022] GET /identity/v3/projects/a841fcca183a4d31ac0320c337760588 => generated 344 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:47.314998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf59aad-06e8-47d9-90af-3c73e424f428 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:47.335828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:47.396430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 209/416] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:47.407455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c803ca7-5455-458e-b2a2-b03b3eb447b9 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:47.407803 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c803ca7-5455-458e-b2a2-b03b3eb447b9 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:47.408064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c803ca7-5455-458e-b2a2-b03b3eb447b9 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:47.409663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 210/417] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:47.415967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1f1bf-431e-4624-9285-f8b0f2b5df56 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:47.416271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1f1bf-431e-4624-9285-f8b0f2b5df56 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:47.416546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1f1bf-431e-4624-9285-f8b0f2b5df56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:47.445720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-93f1f1bf-431e-4624-9285-f8b0f2b5df56 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:47.447307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-93f1f1bf-431e-4624-9285-f8b0f2b5df56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:47.479183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf59aad-06e8-47d9-90af-3c73e424f428 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:47.532929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b02e6-bb29-4959-88aa-97ff4682a242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:47.534613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b02e6-bb29-4959-88aa-97ff4682a242 None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:47.535054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b02e6-bb29-4959-88aa-97ff4682a242 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:47.535963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b02e6-bb29-4959-88aa-97ff4682a242 None admin] PATH_INFO: `/v3/projects/a841fcca183a4d31ac0320c337760588/groups/a618bb16314a48b6aba0809731c489d6/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:47.538515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b02e6-bb29-4959-88aa-97ff4682a242 None admin] RBAC: Authorizing `identity:create_grant(project_id=a841fcca183a4d31ac0320c337760588, group_id=a618bb16314a48b6aba0809731c489d6, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:47.540946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b02e6-bb29-4959-88aa-97ff4682a242 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:47.551709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 211/418] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:47.558087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 208/419] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Aug 1 11:50:47 2022] PUT /identity/v3/projects/a841fcca183a4d31ac0320c337760588/groups/a618bb16314a48b6aba0809731c489d6/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:50:47.559447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1f1bf-431e-4624-9285-f8b0f2b5df56 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:47.597683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b02e6-bb29-4959-88aa-97ff4682a242 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:47.647213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:47.759591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1f1bf-431e-4624-9285-f8b0f2b5df56 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:47.766803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:47.804537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b02e6-bb29-4959-88aa-97ff4682a242 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:47.811069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aad343-28e6-4ad4-9f69-3ab6cbd91c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:47.812518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aad343-28e6-4ad4-9f69-3ab6cbd91c92 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:47.813312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aad343-28e6-4ad4-9f69-3ab6cbd91c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:47.813984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aad343-28e6-4ad4-9f69-3ab6cbd91c92 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:47.836173 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aad343-28e6-4ad4-9f69-3ab6cbd91c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-29aad343-28e6-4ad4-9f69-3ab6cbd91c92 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:47.842533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:47.843658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:47.844541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:47.847674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 212/420] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:47.860142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aad343-28e6-4ad4-9f69-3ab6cbd91c92 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:47.872510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ecc41-c579-415e-bd37-d4a863773b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:47.873487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ecc41-c579-415e-bd37-d4a863773b65 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:47.873941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ecc41-c579-415e-bd37-d4a863773b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:47.874500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ecc41-c579-415e-bd37-d4a863773b65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:47.888190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ecc41-c579-415e-bd37-d4a863773b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-c09ecc41-c579-415e-bd37-d4a863773b65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:47.895677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:47.896733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:47.903194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 209/421] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:47.907604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ecc41-c579-415e-bd37-d4a863773b65 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:48.039320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ecc41-c579-415e-bd37-d4a863773b65 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:48.091941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:48.098796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aad343-28e6-4ad4-9f69-3ab6cbd91c92 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:48.102088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:48.116022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2b19b-9e86-4025-a202-1e599a8150cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:48.116701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2b19b-9e86-4025-a202-1e599a8150cf None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:48.117029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2b19b-9e86-4025-a202-1e599a8150cf None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:48.117302 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b2b19b-9e86-4025-a202-1e599a8150cf None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:48.118142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2b19b-9e86-4025-a202-1e599a8150cf None admin] RBAC: Adding query filter params (name=member) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:48.118447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2b19b-9e86-4025-a202-1e599a8150cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:48.119867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b2b19b-9e86-4025-a202-1e599a8150cf None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:48.138933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 210/422] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:48.147944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2b19b-9e86-4025-a202-1e599a8150cf None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:48.175460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40258cde-913f-455f-93f4-d54752f00be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:48.176014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40258cde-913f-455f-93f4-d54752f00be7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:48.176306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40258cde-913f-455f-93f4-d54752f00be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:48.176604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40258cde-913f-455f-93f4-d54752f00be7 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:48.177340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40258cde-913f-455f-93f4-d54752f00be7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:48.177644 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40258cde-913f-455f-93f4-d54752f00be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:48.179112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40258cde-913f-455f-93f4-d54752f00be7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:48.199139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 213/423] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:48.206322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40258cde-913f-455f-93f4-d54752f00be7 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:48.339592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b2b19b-9e86-4025-a202-1e599a8150cf None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:48.381137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40258cde-913f-455f-93f4-d54752f00be7 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:48.384918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d291c53-2490-4657-91f0-51fedae14304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:48.385475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d291c53-2490-4657-91f0-51fedae14304 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:48.385885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d291c53-2490-4657-91f0-51fedae14304 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:48.386169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d291c53-2490-4657-91f0-51fedae14304 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:48.425107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cd261-4d80-480f-8e1d-53ed1f5b6f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:48.426339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd261-4d80-480f-8e1d-53ed1f5b6f2c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:48.427131 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd261-4d80-480f-8e1d-53ed1f5b6f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:48.427939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd261-4d80-480f-8e1d-53ed1f5b6f2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:48.462119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:48.466803 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:48.482391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d291c53-2490-4657-91f0-51fedae14304 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:48.484477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d291c53-2490-4657-91f0-51fedae14304 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:48.522592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cd261-4d80-480f-8e1d-53ed1f5b6f2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:48.524938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cd261-4d80-480f-8e1d-53ed1f5b6f2c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:48.612520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 211/424] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:48.614004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c759a-2405-4466-91cb-f53980113485 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:48.614355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c759a-2405-4466-91cb-f53980113485 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:48.614699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c759a-2405-4466-91cb-f53980113485 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:48.620505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 212/425] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:48.623857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c759a-2405-4466-91cb-f53980113485 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:48.657672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 214/426] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:48.660047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34bf7e-247e-4819-9202-0877ffa67f01 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:48.660722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34bf7e-247e-4819-9202-0877ffa67f01 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:48.661277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34bf7e-247e-4819-9202-0877ffa67f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:48.694136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ad34bf7e-247e-4819-9202-0877ffa67f01 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:48.696249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ad34bf7e-247e-4819-9202-0877ffa67f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:48.737129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:48.744761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c759a-2405-4466-91cb-f53980113485 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:48.779642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:48.787180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a9431-d287-4e90-ba96-2dda7ac4aec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:48.787965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a9431-d287-4e90-ba96-2dda7ac4aec4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:48.788181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a9431-d287-4e90-ba96-2dda7ac4aec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:48.788510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a9431-d287-4e90-ba96-2dda7ac4aec4 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:48.789729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a9431-d287-4e90-ba96-2dda7ac4aec4 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:48.791199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a9431-d287-4e90-ba96-2dda7ac4aec4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:48.797270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 213/427] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:48 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:48.797270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a9431-d287-4e90-ba96-2dda7ac4aec4 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:48.799280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 215/428] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:48.800137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34bf7e-247e-4819-9202-0877ffa67f01 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:48.934701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a9431-d287-4e90-ba96-2dda7ac4aec4 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:48.948336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34bf7e-247e-4819-9202-0877ffa67f01 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:48.975268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6e7e7-f930-45c6-90d5-b27807822e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:48.976846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6e7e7-f930-45c6-90d5-b27807822e19 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:48.977629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6e7e7-f930-45c6-90d5-b27807822e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:48.978104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6e7e7-f930-45c6-90d5-b27807822e19 None admin] PATH_INFO: `/v3/projects/183e3860f51e45b8828e2c44923ca1f1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:48.980498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6e7e7-f930-45c6-90d5-b27807822e19 None admin] RBAC: Authorizing `identity:get_project(project_id=183e3860f51e45b8828e2c44923ca1f1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:48.982018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed6e7e7-f930-45c6-90d5-b27807822e19 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:48.989509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 214/429] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:48 2022] GET /identity/v3/projects/183e3860f51e45b8828e2c44923ca1f1 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:48.989509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6e7e7-f930-45c6-90d5-b27807822e19 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:48.993064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1800671-27f0-4db7-bce5-0b8f7cbd6c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:48.993750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1800671-27f0-4db7-bce5-0b8f7cbd6c63 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:48.994004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1800671-27f0-4db7-bce5-0b8f7cbd6c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:48.994273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1800671-27f0-4db7-bce5-0b8f7cbd6c63 None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:48.996317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1800671-27f0-4db7-bce5-0b8f7cbd6c63 None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:48.996989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1800671-27f0-4db7-bce5-0b8f7cbd6c63 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:48.998629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 216/430] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:48 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:49.002350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1800671-27f0-4db7-bce5-0b8f7cbd6c63 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:49.036491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:49.069148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:49.118231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1800671-27f0-4db7-bce5-0b8f7cbd6c63 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:49.136721 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6e7e7-f930-45c6-90d5-b27807822e19 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:49.156207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7a06c-7106-448f-8f24-fa61541b3bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:49.156984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7a06c-7106-448f-8f24-fa61541b3bad None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:49.157363 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7a06c-7106-448f-8f24-fa61541b3bad None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:49.157638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7a06c-7106-448f-8f24-fa61541b3bad None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:49.158963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7a06c-7106-448f-8f24-fa61541b3bad None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:49.160832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7a06c-7106-448f-8f24-fa61541b3bad None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:49.167555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 217/431] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:48 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:49.167555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7a06c-7106-448f-8f24-fa61541b3bad None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:49.189005 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c2e1-b5f4-4ec3-9546-2da66e80e6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:49.189948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c2e1-b5f4-4ec3-9546-2da66e80e6dd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:49.190367 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c2e1-b5f4-4ec3-9546-2da66e80e6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:49.191288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e1c2e1-b5f4-4ec3-9546-2da66e80e6dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:49.205667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e1c2e1-b5f4-4ec3-9546-2da66e80e6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-72e1c2e1-b5f4-4ec3-9546-2da66e80e6dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:49.212139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:49.213217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 01 11:50:49.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:49.216725 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 215/432] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:49.218378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c2e1-b5f4-4ec3-9546-2da66e80e6dd None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:49.292181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7a06c-7106-448f-8f24-fa61541b3bad None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:49.326946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:49.336674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c02214-1bd6-4059-9852-4c3bb9137561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:49.337545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c02214-1bd6-4059-9852-4c3bb9137561 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:49.338339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c02214-1bd6-4059-9852-4c3bb9137561 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:49.338777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c02214-1bd6-4059-9852-4c3bb9137561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:49.340054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c02214-1bd6-4059-9852-4c3bb9137561 None admin] RBAC: Adding query filter params (group.id=af2b9bb190684cea998daed0b59c0ea4, role.id=4f5140de9df4440a8fb63d93249d14d1, scope.project.id=183e3860f51e45b8828e2c44923ca1f1) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:49.340815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c02214-1bd6-4059-9852-4c3bb9137561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:49.342597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c02214-1bd6-4059-9852-4c3bb9137561 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:49.350187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:49.350942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 218/433] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Aug 1 11:50:49 2022] GET /identity/v3/role_assignments?group.id=af2b9bb190684cea998daed0b59c0ea4&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:49.352060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c02214-1bd6-4059-9852-4c3bb9137561 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:49.384281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e1c2e1-b5f4-4ec3-9546-2da66e80e6dd None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:49.457133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44323014-d1cd-414c-85a8-1cc37002788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:49.458246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44323014-d1cd-414c-85a8-1cc37002788e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:49.458364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44323014-d1cd-414c-85a8-1cc37002788e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:49.458727 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44323014-d1cd-414c-85a8-1cc37002788e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:49.459734 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44323014-d1cd-414c-85a8-1cc37002788e None admin] RBAC: Adding query filter params (group.id=af2b9bb190684cea998daed0b59c0ea4, role.id=8e13cd96bb124d52b5c8883fe960424f, scope.project.id=183e3860f51e45b8828e2c44923ca1f1) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:49.460434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44323014-d1cd-414c-85a8-1cc37002788e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:49.461880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44323014-d1cd-414c-85a8-1cc37002788e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:49.484334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 216/434] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Aug 1 11:50:49 2022] GET /identity/v3/role_assignments?group.id=af2b9bb190684cea998daed0b59c0ea4&scope.project.id=183e3860f51e45b8828e2c44923ca1f1&role.id=8e13cd96bb124d52b5c8883fe960424f => generated 611 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:49.494426 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c02214-1bd6-4059-9852-4c3bb9137561 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:49.536184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08683455-dc72-4dc2-8d79-7e811f89220d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:49.537607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08683455-dc72-4dc2-8d79-7e811f89220d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:49.538401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08683455-dc72-4dc2-8d79-7e811f89220d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:49.539147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08683455-dc72-4dc2-8d79-7e811f89220d None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:49.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08683455-dc72-4dc2-8d79-7e811f89220d None admin] RBAC: Adding query filter params (name=member) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:49.541622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08683455-dc72-4dc2-8d79-7e811f89220d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:49.543833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08683455-dc72-4dc2-8d79-7e811f89220d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:49.555098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 219/435] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:50:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:49.563260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44323014-d1cd-414c-85a8-1cc37002788e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:49.679323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31282bc4-fdea-4763-9936-f083d4fb3cdc None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:49.679852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31282bc4-fdea-4763-9936-f083d4fb3cdc None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:49.679852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31282bc4-fdea-4763-9936-f083d4fb3cdc None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:49.681589 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 220/436] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:49.687317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7066-d52a-41c1-a9bc-fd8d70c75f04 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:49.687653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7066-d52a-41c1-a9bc-fd8d70c75f04 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:49.687874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb7066-d52a-41c1-a9bc-fd8d70c75f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:49.708823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-83fb7066-d52a-41c1-a9bc-fd8d70c75f04 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:49.710130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-83fb7066-d52a-41c1-a9bc-fd8d70c75f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:49.730869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44323014-d1cd-414c-85a8-1cc37002788e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:49.783928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:49.789386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a867364-c0e1-4126-a71a-f0a4365237d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:49.790034 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a867364-c0e1-4126-a71a-f0a4365237d1 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:49.790318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a867364-c0e1-4126-a71a-f0a4365237d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:49.790599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a867364-c0e1-4126-a71a-f0a4365237d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:49.814471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 221/437] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:49.815852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481789e5-94b0-4842-8ae2-3c68e7dee9f0 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:49.816457 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481789e5-94b0-4842-8ae2-3c68e7dee9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:49.820435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481789e5-94b0-4842-8ae2-3c68e7dee9f0 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:49.821946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 222/438] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:49.822413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:49.829854 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11026b9-41b2-4b53-92ef-b29d685b4d7c None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:49.831013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11026b9-41b2-4b53-92ef-b29d685b4d7c None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:49.831700 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11026b9-41b2-4b53-92ef-b29d685b4d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:49.854804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d11026b9-41b2-4b53-92ef-b29d685b4d7c None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:49.855983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d11026b9-41b2-4b53-92ef-b29d685b4d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:49.898924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a867364-c0e1-4126-a71a-f0a4365237d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:49.900267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a867364-c0e1-4126-a71a-f0a4365237d1 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:49.945433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 223/439] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:49.946430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11026b9-41b2-4b53-92ef-b29d685b4d7c None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:50.027325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 217/440] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:50.029341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a867364-c0e1-4126-a71a-f0a4365237d1 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:50.075310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11026b9-41b2-4b53-92ef-b29d685b4d7c None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:50.097851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:50.101471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:50.139966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166308a0-2bc8-4e41-9a74-38cb9dfcfaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:50.141133 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166308a0-2bc8-4e41-9a74-38cb9dfcfaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:50.142382 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166308a0-2bc8-4e41-9a74-38cb9dfcfaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:50.142668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166308a0-2bc8-4e41-9a74-38cb9dfcfaa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:50.157590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166308a0-2bc8-4e41-9a74-38cb9dfcfaa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-166308a0-2bc8-4e41-9a74-38cb9dfcfaa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:50.163096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:50.164584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 224/441] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:50.165620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166308a0-2bc8-4e41-9a74-38cb9dfcfaa5 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:50.181117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a867364-c0e1-4126-a71a-f0a4365237d1 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:50.220698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb213cd9-7257-47e4-924f-d5e806d9b967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:50.221435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb213cd9-7257-47e4-924f-d5e806d9b967 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:50.221634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb213cd9-7257-47e4-924f-d5e806d9b967 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:50.221909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb213cd9-7257-47e4-924f-d5e806d9b967 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:50.233654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb213cd9-7257-47e4-924f-d5e806d9b967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-cb213cd9-7257-47e4-924f-d5e806d9b967 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:50.239156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:50.240138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:50.241385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 218/442] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:50:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:50.242361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb213cd9-7257-47e4-924f-d5e806d9b967 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:50.288247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166308a0-2bc8-4e41-9a74-38cb9dfcfaa5 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:50.323626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323abdc9-74b2-4272-9868-4cdfa287ba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:50.324276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323abdc9-74b2-4272-9868-4cdfa287ba8b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:50.324588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323abdc9-74b2-4272-9868-4cdfa287ba8b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:50.324853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323abdc9-74b2-4272-9868-4cdfa287ba8b None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:50.326036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323abdc9-74b2-4272-9868-4cdfa287ba8b None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:50.327505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323abdc9-74b2-4272-9868-4cdfa287ba8b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:50.329264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 225/443] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:50 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:50.330036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323abdc9-74b2-4272-9868-4cdfa287ba8b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:50.358001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb213cd9-7257-47e4-924f-d5e806d9b967 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:50.378404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:50.381286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:50.406149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be715cab-f50e-417e-92b5-35c2d889de9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:50.406803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be715cab-f50e-417e-92b5-35c2d889de9c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:50.407083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be715cab-f50e-417e-92b5-35c2d889de9c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:50.407350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be715cab-f50e-417e-92b5-35c2d889de9c None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:50.408075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be715cab-f50e-417e-92b5-35c2d889de9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:50.408413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be715cab-f50e-417e-92b5-35c2d889de9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:50.409865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be715cab-f50e-417e-92b5-35c2d889de9c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:50.418502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 219/444] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:50.419420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be715cab-f50e-417e-92b5-35c2d889de9c None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:50.447785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323abdc9-74b2-4272-9868-4cdfa287ba8b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:50.483716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd5db8-2edb-45ad-bafd-4e4855d28f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:50.484359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5db8-2edb-45ad-bafd-4e4855d28f74 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:50.484648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5db8-2edb-45ad-bafd-4e4855d28f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:50.484918 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd5db8-2edb-45ad-bafd-4e4855d28f74 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:50.485652 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd5db8-2edb-45ad-bafd-4e4855d28f74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:50.485958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd5db8-2edb-45ad-bafd-4e4855d28f74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:50.487725 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd5db8-2edb-45ad-bafd-4e4855d28f74 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:50.496950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 226/445] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:50.497788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd5db8-2edb-45ad-bafd-4e4855d28f74 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:50.527161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be715cab-f50e-417e-92b5-35c2d889de9c None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:50.570717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d0854-a8f1-4576-8988-fe9c4f45bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:50.571581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d0854-a8f1-4576-8988-fe9c4f45bd3d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:50.571865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d0854-a8f1-4576-8988-fe9c4f45bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:50.572166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d0854-a8f1-4576-8988-fe9c4f45bd3d None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:50.573530 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d0854-a8f1-4576-8988-fe9c4f45bd3d None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:50.575000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d0854-a8f1-4576-8988-fe9c4f45bd3d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:50.581963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 220/446] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:50 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:50.582835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d0854-a8f1-4576-8988-fe9c4f45bd3d None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:50.609522 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd5db8-2edb-45ad-bafd-4e4855d28f74 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:50.613067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:50.631512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:50.648771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741502b-ae7e-4e67-b6e6-9ae2e9cc2792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:50.649405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741502b-ae7e-4e67-b6e6-9ae2e9cc2792 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:50.649724 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741502b-ae7e-4e67-b6e6-9ae2e9cc2792 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:50.650002 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741502b-ae7e-4e67-b6e6-9ae2e9cc2792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:50.693427 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d0854-a8f1-4576-8988-fe9c4f45bd3d None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:50.721997 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741502b-ae7e-4e67-b6e6-9ae2e9cc2792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:50.723465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741502b-ae7e-4e67-b6e6-9ae2e9cc2792 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:50.730483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba23fa-4bda-4e81-9902-97c27f28611b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:50.731124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba23fa-4bda-4e81-9902-97c27f28611b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:50.731635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba23fa-4bda-4e81-9902-97c27f28611b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:50.731715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba23fa-4bda-4e81-9902-97c27f28611b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:50.802303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba23fa-4bda-4e81-9902-97c27f28611b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:50.803845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba23fa-4bda-4e81-9902-97c27f28611b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:50.838789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 227/447] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:50.838789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741502b-ae7e-4e67-b6e6-9ae2e9cc2792 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:50.838789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:50.865743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:50.928524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 221/448] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:50.933337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba23fa-4bda-4e81-9902-97c27f28611b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:50.948721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741502b-ae7e-4e67-b6e6-9ae2e9cc2792 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:50.983888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478902c-f35a-4fc0-843b-9139d6e0c37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:50.984543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478902c-f35a-4fc0-843b-9139d6e0c37b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:50.984828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478902c-f35a-4fc0-843b-9139d6e0c37b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:50.985101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478902c-f35a-4fc0-843b-9139d6e0c37b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:50.985907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478902c-f35a-4fc0-843b-9139d6e0c37b None admin] RBAC: Adding query filter params (group.id=af2b9bb190684cea998daed0b59c0ea4, role.id=4f5140de9df4440a8fb63d93249d14d1, scope.project.id=876fcd5989ee455a89866281d214740e) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:50.986569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478902c-f35a-4fc0-843b-9139d6e0c37b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:50.988022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478902c-f35a-4fc0-843b-9139d6e0c37b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:50.994280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 228/449] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Aug 1 11:50:50 2022] GET /identity/v3/role_assignments?group.id=af2b9bb190684cea998daed0b59c0ea4&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=4f5140de9df4440a8fb63d93249d14d1 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:50.995101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478902c-f35a-4fc0-843b-9139d6e0c37b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:51.067706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba23fa-4bda-4e81-9902-97c27f28611b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:51.080006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:51.114162 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33f303-e861-4400-9fc5-5b0d5dabcc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:51.114832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33f303-e861-4400-9fc5-5b0d5dabcc16 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:51.115124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33f303-e861-4400-9fc5-5b0d5dabcc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:51.115439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33f303-e861-4400-9fc5-5b0d5dabcc16 None admin] PATH_INFO: `/v3/projects/876fcd5989ee455a89866281d214740e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:51.116741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33f303-e861-4400-9fc5-5b0d5dabcc16 None admin] RBAC: Authorizing `identity:get_project(project_id=876fcd5989ee455a89866281d214740e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:51.118286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33f303-e861-4400-9fc5-5b0d5dabcc16 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:51.120497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 222/450] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:50 2022] GET /identity/v3/projects/876fcd5989ee455a89866281d214740e => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:51.127697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33f303-e861-4400-9fc5-5b0d5dabcc16 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:51.136881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478902c-f35a-4fc0-843b-9139d6e0c37b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:51.150641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:51.176871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a83ff-fd41-46c3-b1c7-3fe2a0ef1ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:51.177531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a83ff-fd41-46c3-b1c7-3fe2a0ef1ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:51.177914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a83ff-fd41-46c3-b1c7-3fe2a0ef1ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:51.178189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a83ff-fd41-46c3-b1c7-3fe2a0ef1ba1 None admin] PATH_INFO: `/v3/projects/a841fcca183a4d31ac0320c337760588` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:51.179805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a83ff-fd41-46c3-b1c7-3fe2a0ef1ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=a841fcca183a4d31ac0320c337760588)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:51.181107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a83ff-fd41-46c3-b1c7-3fe2a0ef1ba1 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:51.183053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 229/451] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:50:50 2022] GET /identity/v3/projects/a841fcca183a4d31ac0320c337760588 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:51.189039 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a83ff-fd41-46c3-b1c7-3fe2a0ef1ba1 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:51.274992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33f303-e861-4400-9fc5-5b0d5dabcc16 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:51.321391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985c6f4-89aa-4b0c-9303-3277b95afa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:51.322080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985c6f4-89aa-4b0c-9303-3277b95afa56 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:51.322362 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985c6f4-89aa-4b0c-9303-3277b95afa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:51.322631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6985c6f4-89aa-4b0c-9303-3277b95afa56 None admin] PATH_INFO: `/v3/groups/af2b9bb190684cea998daed0b59c0ea4` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:51.325361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985c6f4-89aa-4b0c-9303-3277b95afa56 None admin] RBAC: Authorizing `identity:get_group(group_id=af2b9bb190684cea998daed0b59c0ea4)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:51.326881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6985c6f4-89aa-4b0c-9303-3277b95afa56 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:51.328673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 223/452] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:51 2022] GET /identity/v3/groups/af2b9bb190684cea998daed0b59c0ea4 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:51.330473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a83ff-fd41-46c3-b1c7-3fe2a0ef1ba1 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:51.334917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985c6f4-89aa-4b0c-9303-3277b95afa56 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:51.378104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73a1a4-da69-40f2-804f-dafc6a875014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:51.378688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a1a4-da69-40f2-804f-dafc6a875014 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:51.378962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a1a4-da69-40f2-804f-dafc6a875014 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:51.379234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f73a1a4-da69-40f2-804f-dafc6a875014 None admin] PATH_INFO: `/v3/groups/a618bb16314a48b6aba0809731c489d6` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:51.380920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73a1a4-da69-40f2-804f-dafc6a875014 None admin] RBAC: Authorizing `identity:get_group(group_id=a618bb16314a48b6aba0809731c489d6)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:51.382142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f73a1a4-da69-40f2-804f-dafc6a875014 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:51.384023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 230/453] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Aug 1 11:50:51 2022] GET /identity/v3/groups/a618bb16314a48b6aba0809731c489d6 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:51.390181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73a1a4-da69-40f2-804f-dafc6a875014 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:51.394675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:51.430901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:51.459211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6985c6f4-89aa-4b0c-9303-3277b95afa56 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:51.496864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89aa11-4140-43f1-ad31-bd91cec8f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:51.497636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89aa11-4140-43f1-ad31-bd91cec8f2ac None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:51.498045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89aa11-4140-43f1-ad31-bd91cec8f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:51.498434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89aa11-4140-43f1-ad31-bd91cec8f2ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:51.499347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89aa11-4140-43f1-ad31-bd91cec8f2ac None admin] RBAC: Adding query filter params (group.id=af2b9bb190684cea998daed0b59c0ea4, role.id=8e13cd96bb124d52b5c8883fe960424f, scope.project.id=876fcd5989ee455a89866281d214740e) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:51.499743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89aa11-4140-43f1-ad31-bd91cec8f2ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:51.501359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89aa11-4140-43f1-ad31-bd91cec8f2ac None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:51.501859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f73a1a4-da69-40f2-804f-dafc6a875014 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:51.507757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 224/454] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Aug 1 11:50:51 2022] GET /identity/v3/role_assignments?group.id=af2b9bb190684cea998daed0b59c0ea4&scope.project.id=876fcd5989ee455a89866281d214740e&role.id=8e13cd96bb124d52b5c8883fe960424f => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:51.545396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f33b0a-2d42-4e4f-ba2f-566d85497d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:51.546608 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f33b0a-2d42-4e4f-ba2f-566d85497d02 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:51.547057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f33b0a-2d42-4e4f-ba2f-566d85497d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:51.547360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f33b0a-2d42-4e4f-ba2f-566d85497d02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:51.549663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f33b0a-2d42-4e4f-ba2f-566d85497d02 None admin] RBAC: Adding query filter params (group.id=a618bb16314a48b6aba0809731c489d6, role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=a841fcca183a4d31ac0320c337760588) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:51.549975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f33b0a-2d42-4e4f-ba2f-566d85497d02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:51.552020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f33b0a-2d42-4e4f-ba2f-566d85497d02 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:51.558634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 231/455] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Aug 1 11:50:51 2022] GET /identity/v3/role_assignments?group.id=a618bb16314a48b6aba0809731c489d6&scope.project.id=a841fcca183a4d31ac0320c337760588&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:54.106286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02534db1-3c37-448f-ab7d-7b99d9e8f612 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.107297 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02534db1-3c37-448f-ab7d-7b99d9e8f612 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.107788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02534db1-3c37-448f-ab7d-7b99d9e8f612 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.109413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 225/456] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:54.115437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade51a2-4668-4bf2-b882-d466fc92fbe3 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.116240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade51a2-4668-4bf2-b882-d466fc92fbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.116757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade51a2-4668-4bf2-b882-d466fc92fbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.141382 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-eade51a2-4668-4bf2-b882-d466fc92fbe3 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:54.142602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-eade51a2-4668-4bf2-b882-d466fc92fbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:54.240575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:54.247884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 232/457] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:54.283216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02534db1-3c37-448f-ab7d-7b99d9e8f612 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:54.422959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02534db1-3c37-448f-ab7d-7b99d9e8f612 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:54.462918 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9941d34-356e-4f6a-a52b-cbc5523a48f0 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.463255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9941d34-356e-4f6a-a52b-cbc5523a48f0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.463485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9941d34-356e-4f6a-a52b-cbc5523a48f0 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.463653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7191-092e-439a-acd2-3c7732fb11fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:54.464267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7191-092e-439a-acd2-3c7732fb11fe None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.464572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7191-092e-439a-acd2-3c7732fb11fe None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.464845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7191-092e-439a-acd2-3c7732fb11fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.466213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 233/458] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:54.472645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71705f29-8ed2-4cf0-afd1-4a41f16f6021 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.473190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71705f29-8ed2-4cf0-afd1-4a41f16f6021 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.473655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71705f29-8ed2-4cf0-afd1-4a41f16f6021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.483206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba7191-092e-439a-acd2-3c7732fb11fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:54.484041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba7191-092e-439a-acd2-3c7732fb11fe None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:54.495786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-71705f29-8ed2-4cf0-afd1-4a41f16f6021 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:54.496632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:54.497533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-71705f29-8ed2-4cf0-afd1-4a41f16f6021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-92ba7191-092e-439a-acd2-3c7732fb11fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:50:54.509633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:54.510537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:54.511380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:50:54.511380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:50:54.511380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:54.511380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:54.516794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 226/459] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:54.535152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7191-092e-439a-acd2-3c7732fb11fe None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:54.586566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 234/460] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:54.641811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca293193-9a4b-434a-982b-1aad34bb6db6 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.642170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca293193-9a4b-434a-982b-1aad34bb6db6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.642371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca293193-9a4b-434a-982b-1aad34bb6db6 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.643992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 235/461] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:50:54.649238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19438c6-1a1b-4d71-b35c-4432dac760d6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.649532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19438c6-1a1b-4d71-b35c-4432dac760d6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.649844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19438c6-1a1b-4d71-b35c-4432dac760d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.674819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-f19438c6-1a1b-4d71-b35c-4432dac760d6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:54.675935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f19438c6-1a1b-4d71-b35c-4432dac760d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:54.690199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7191-092e-439a-acd2-3c7732fb11fe None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:54.726589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220d3a4-25b5-44fa-90a4-e3fee3252264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:54.727611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220d3a4-25b5-44fa-90a4-e3fee3252264 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.728443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220d3a4-25b5-44fa-90a4-e3fee3252264 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.728975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220d3a4-25b5-44fa-90a4-e3fee3252264 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.729917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220d3a4-25b5-44fa-90a4-e3fee3252264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:54.730425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220d3a4-25b5-44fa-90a4-e3fee3252264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:54.732020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220d3a4-25b5-44fa-90a4-e3fee3252264 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:54.751610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 227/463] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:54.756399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 236/463] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:54.757702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220d3a4-25b5-44fa-90a4-e3fee3252264 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:54.758044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19438c6-1a1b-4d71-b35c-4432dac760d6 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:54.819033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:54.832342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:54.876804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220d3a4-25b5-44fa-90a4-e3fee3252264 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:54.877122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19438c6-1a1b-4d71-b35c-4432dac760d6 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:54.913788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee2348-cd59-46fc-b4b5-3d151bb0aea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:54.914042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec266b4-6c6f-430d-a449-786ef3154ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:54.914417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee2348-cd59-46fc-b4b5-3d151bb0aea1 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.914513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec266b4-6c6f-430d-a449-786ef3154ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.914701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee2348-cd59-46fc-b4b5-3d151bb0aea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.914809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec266b4-6c6f-430d-a449-786ef3154ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.914980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec266b4-6c6f-430d-a449-786ef3154ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.915082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee2348-cd59-46fc-b4b5-3d151bb0aea1 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.915655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee2348-cd59-46fc-b4b5-3d151bb0aea1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:54.917137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee2348-cd59-46fc-b4b5-3d151bb0aea1 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:54.926263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec266b4-6c6f-430d-a449-786ef3154ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:54.927774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec266b4-6c6f-430d-a449-786ef3154ec6 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:54.948326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 237/464] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:50:54 2022] POST /identity/v3/users => generated 246 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:54.948326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee2348-cd59-46fc-b4b5-3d151bb0aea1 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-9ec266b4-6c6f-430d-a449-786ef3154ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:50:54.948584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:54.949384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:54.950197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:50:54.950197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:50:54.950197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:54.950197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:54.950197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 228/465] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:54.950197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab3c1-73c2-4c57-8163-9e7c3a284062 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:54.950197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab3c1-73c2-4c57-8163-9e7c3a284062 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:54.950197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab3c1-73c2-4c57-8163-9e7c3a284062 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:54.952540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 229/466] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:54.953477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdab3c1-73c2-4c57-8163-9e7c3a284062 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:55.070325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee2348-cd59-46fc-b4b5-3d151bb0aea1 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:55.075174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdab3c1-73c2-4c57-8163-9e7c3a284062 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:55.084330 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:55.087809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:55.113874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372b4ee-2732-42c6-b405-d3915961eb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:55.114537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372b4ee-2732-42c6-b405-d3915961eb9a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:55.114820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372b4ee-2732-42c6-b405-d3915961eb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:55.115239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372b4ee-2732-42c6-b405-d3915961eb9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:55.123148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02c4cc-6c64-4721-9985-1263ceddcc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:55.124648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02c4cc-6c64-4721-9985-1263ceddcc8a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:55.125102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02c4cc-6c64-4721-9985-1263ceddcc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:55.125633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02c4cc-6c64-4721-9985-1263ceddcc8a None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:55.126527 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02c4cc-6c64-4721-9985-1263ceddcc8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:55.127064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02c4cc-6c64-4721-9985-1263ceddcc8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:55.127869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372b4ee-2732-42c6-b405-d3915961eb9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:55.129249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02c4cc-6c64-4721-9985-1263ceddcc8a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:55.129445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372b4ee-2732-42c6-b405-d3915961eb9a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:55.145198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 230/467] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:55.146568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2985c-6c62-4f96-9abb-b1f31184d6fc None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:55.146853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2985c-6c62-4f96-9abb-b1f31184d6fc None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:55.147125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2985c-6c62-4f96-9abb-b1f31184d6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-0372b4ee-2732-42c6-b405-d3915961eb9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:50:55.147960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:55.148936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:55.149779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:50:55.149779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:50:55.149779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:55.149779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:55.149779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 238/468] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:55.154762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372b4ee-2732-42c6-b405-d3915961eb9a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:55.176567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9fb2985c-6c62-4f96-9abb-b1f31184d6fc None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:55.177740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9fb2985c-6c62-4f96-9abb-b1f31184d6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:55.266018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 231/469] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:55.266798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2985c-6c62-4f96-9abb-b1f31184d6fc None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:55.293286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372b4ee-2732-42c6-b405-d3915961eb9a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:55.331844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceda2e-6cb1-4f4d-ae41-bfe937d88574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:55.332525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceda2e-6cb1-4f4d-ae41-bfe937d88574 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:55.332863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceda2e-6cb1-4f4d-ae41-bfe937d88574 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:55.333140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceda2e-6cb1-4f4d-ae41-bfe937d88574 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:55.333974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceda2e-6cb1-4f4d-ae41-bfe937d88574 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:55.335447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceda2e-6cb1-4f4d-ae41-bfe937d88574 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:55.362042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 239/470] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:50:55 2022] POST /identity/v3/users => generated 248 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 11:50:55.365030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceda2e-6cb1-4f4d-ae41-bfe937d88574 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:55.417909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:55.430151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2985c-6c62-4f96-9abb-b1f31184d6fc None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:55.433694 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:55.503464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2b6a8-3bd8-4a00-bc35-abc9130243c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:55.504214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2b6a8-3bd8-4a00-bc35-abc9130243c9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:55.504515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2b6a8-3bd8-4a00-bc35-abc9130243c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:55.504783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2b6a8-3bd8-4a00-bc35-abc9130243c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:55.505539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2b6a8-3bd8-4a00-bc35-abc9130243c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:55.505899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2b6a8-3bd8-4a00-bc35-abc9130243c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:55.507518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2b6a8-3bd8-4a00-bc35-abc9130243c9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:55.521078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 232/471] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:55.527379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edea5-ddd8-4f47-ab60-3f151c7e32fc None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:55.527875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edea5-ddd8-4f47-ab60-3f151c7e32fc None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:55.528157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edea5-ddd8-4f47-ab60-3f151c7e32fc None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:55.532345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 233/472] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:55.532345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9edea5-ddd8-4f47-ab60-3f151c7e32fc None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:55.557493 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceda2e-6cb1-4f4d-ae41-bfe937d88574 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:55.603942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8cc9c-53a5-4e55-bd33-0af0915b0656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:55.604624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8cc9c-53a5-4e55-bd33-0af0915b0656 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:55.604904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8cc9c-53a5-4e55-bd33-0af0915b0656 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:55.605175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8cc9c-53a5-4e55-bd33-0af0915b0656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:55.617551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8cc9c-53a5-4e55-bd33-0af0915b0656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:55.619083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8cc9c-53a5-4e55-bd33-0af0915b0656 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-c6f8cc9c-53a5-4e55-bd33-0af0915b0656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:50:55.634736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:55.635751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:55.636790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:50:55.636790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:50:55.636790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:55.636790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:55.636790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 240/473] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:55.637453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc1d38-b237-408d-9c3d-52d1becac059 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:55.637910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc1d38-b237-408d-9c3d-52d1becac059 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:55.638415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc1d38-b237-408d-9c3d-52d1becac059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:55.666255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9edea5-ddd8-4f47-ab60-3f151c7e32fc None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:55.668516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-f7bc1d38-b237-408d-9c3d-52d1becac059 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:55.670638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f7bc1d38-b237-408d-9c3d-52d1becac059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:55.715659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e28523-fcaf-48f5-b32c-55a8981082e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:55.716660 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e28523-fcaf-48f5-b32c-55a8981082e2 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:55.717683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e28523-fcaf-48f5-b32c-55a8981082e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:55.718182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e28523-fcaf-48f5-b32c-55a8981082e2 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:55.719608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e28523-fcaf-48f5-b32c-55a8981082e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:55.722082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e28523-fcaf-48f5-b32c-55a8981082e2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:55.754898 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 234/474] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:50:55 2022] POST /identity/v3/users => generated 248 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:55.756361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e28523-fcaf-48f5-b32c-55a8981082e2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:55.764771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 241/475] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:55.779953 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:55.805839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc1d38-b237-408d-9c3d-52d1becac059 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:55.809484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:55.962402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e28523-fcaf-48f5-b32c-55a8981082e2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:56.011249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0705a5-0011-49ee-a3eb-14d1493c49c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:56.011783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc1d38-b237-408d-9c3d-52d1becac059 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:56.011893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0705a5-0011-49ee-a3eb-14d1493c49c3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:56.012069 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0705a5-0011-49ee-a3eb-14d1493c49c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:56.013113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0705a5-0011-49ee-a3eb-14d1493c49c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:56.013835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0705a5-0011-49ee-a3eb-14d1493c49c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:56.015125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0705a5-0011-49ee-a3eb-14d1493c49c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:56.015688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0705a5-0011-49ee-a3eb-14d1493c49c3 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:56.029446 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 235/476] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:56.035008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0705a5-0011-49ee-a3eb-14d1493c49c3 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:56.060977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd1dec-8c29-482d-8cf1-c043db9af997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:56.061943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd1dec-8c29-482d-8cf1-c043db9af997 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:56.062360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd1dec-8c29-482d-8cf1-c043db9af997 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:56.062772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd1dec-8c29-482d-8cf1-c043db9af997 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:56.063658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd1dec-8c29-482d-8cf1-c043db9af997 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:56.066009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd1dec-8c29-482d-8cf1-c043db9af997 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:56.077488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.sql.core [None req-81dd1dec-8c29-482d-8cf1-c043db9af997 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 11:50:56.077488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 11:50:56.077488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [parameters: {'id': '7b7486375b464d54a72c285ee62aa3a6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 11:50:56.077488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80163) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-81dd1dec-8c29-482d-8cf1-c043db9af997 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application conn.query(q) Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application result.read() Aug 01 11:50:56.100604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return ref.to_dict() Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 11:50:56.101592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application next(self.gen) Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application yield resource Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application next(self.gen) Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise self.value Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application session.commit() Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 11:50:56.102488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self.session.flush() Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application compat.raise_( Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 11:50:56.103320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 11:50:56.104182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application conn.query(q) Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application result.read() Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 11:50:56.105044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [parameters: {'id': '7b7486375b464d54a72c285ee62aa3a6', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:56.105808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 01 11:50:56.106587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:56.107484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 242/477] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Aug 1 11:50:55 2022] POST /identity/v3/roles => generated 144 bytes in 301 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:56.112328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd1dec-8c29-482d-8cf1-c043db9af997 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:56.133677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:56.175874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0705a5-0011-49ee-a3eb-14d1493c49c3 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:56.220666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:56.231152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3da51-6854-41ee-8edf-2605a6907f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:56.231807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3da51-6854-41ee-8edf-2605a6907f25 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:56.232087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3da51-6854-41ee-8edf-2605a6907f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:56.232388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3da51-6854-41ee-8edf-2605a6907f25 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:56.233389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3da51-6854-41ee-8edf-2605a6907f25 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:56.237172 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce3da51-6854-41ee-8edf-2605a6907f25 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:56.267314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 236/478] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:50:56 2022] POST /identity/v3/users => generated 249 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:50:56.279325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd1dec-8c29-482d-8cf1-c043db9af997 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:56.323964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d76a3e5-ecb9-4950-90d9-482b17f13b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:56.325216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76a3e5-ecb9-4950-90d9-482b17f13b87 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:56.325754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76a3e5-ecb9-4950-90d9-482b17f13b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:56.326242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76a3e5-ecb9-4950-90d9-482b17f13b87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:56.340533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d76a3e5-ecb9-4950-90d9-482b17f13b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-6d76a3e5-ecb9-4950-90d9-482b17f13b87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:56.346251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:56.347156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 01 11:50:56.347975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:56.350930 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 243/479] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:50:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:56.357481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3da51-6854-41ee-8edf-2605a6907f25 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:56.527841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3da51-6854-41ee-8edf-2605a6907f25 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:56.556672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:56.567101 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92fcc2-588e-4394-a836-45a258a2b90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:56.568112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92fcc2-588e-4394-a836-45a258a2b90c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:56.568455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92fcc2-588e-4394-a836-45a258a2b90c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:56.568730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92fcc2-588e-4394-a836-45a258a2b90c None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:56.569465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92fcc2-588e-4394-a836-45a258a2b90c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:56.569769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92fcc2-588e-4394-a836-45a258a2b90c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:56.571143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92fcc2-588e-4394-a836-45a258a2b90c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:56.579859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 237/480] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:50:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:57.265378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38b9a7-b164-4540-9066-7d1b749b5826 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:57.265882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38b9a7-b164-4540-9066-7d1b749b5826 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:57.265882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38b9a7-b164-4540-9066-7d1b749b5826 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:57.267619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 244/481] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:57.278005 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe533ddf-431b-4b35-a2e7-9cbadfb33f2b None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:57.278502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe533ddf-431b-4b35-a2e7-9cbadfb33f2b None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:57.278578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe533ddf-431b-4b35-a2e7-9cbadfb33f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:57.303061 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-fe533ddf-431b-4b35-a2e7-9cbadfb33f2b None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:57.304270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-fe533ddf-431b-4b35-a2e7-9cbadfb33f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:57.398503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 238/482] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:57.430498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38b9a7-b164-4540-9066-7d1b749b5826 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:57.554071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38b9a7-b164-4540-9066-7d1b749b5826 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:57.581180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:57.592960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b22393-f8c9-4afd-8648-b41db38d7673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:57.593626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b22393-f8c9-4afd-8648-b41db38d7673 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:57.593907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b22393-f8c9-4afd-8648-b41db38d7673 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:57.594180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b22393-f8c9-4afd-8648-b41db38d7673 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:57.606676 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b22393-f8c9-4afd-8648-b41db38d7673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-70b22393-f8c9-4afd-8648-b41db38d7673 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:57.612227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:57.613113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:50:57.613897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:57.616531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 245/483] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:50:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:57.624152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe533ddf-431b-4b35-a2e7-9cbadfb33f2b None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:57.755689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe533ddf-431b-4b35-a2e7-9cbadfb33f2b None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:57.787020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:57.810461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f9647-e320-4611-8bd5-1b88ce80d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:57.811026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f9647-e320-4611-8bd5-1b88ce80d487 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:57.811303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f9647-e320-4611-8bd5-1b88ce80d487 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:57.811574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f9647-e320-4611-8bd5-1b88ce80d487 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:57.812321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f9647-e320-4611-8bd5-1b88ce80d487 None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:57.812634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f9647-e320-4611-8bd5-1b88ce80d487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:57.814023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f9647-e320-4611-8bd5-1b88ce80d487 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:57.833290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 239/484] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:50:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:57.838967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b22393-f8c9-4afd-8648-b41db38d7673 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:57.947094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92fd8f-84dd-481e-af2f-0998af4feb97 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:57.947424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92fd8f-84dd-481e-af2f-0998af4feb97 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:57.947644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92fd8f-84dd-481e-af2f-0998af4feb97 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:57.949312 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 240/485] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:57.957050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4c17c-0bb5-41e8-a858-4b588d3bb4f3 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:57.957050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4c17c-0bb5-41e8-a858-4b588d3bb4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:57.957050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4c17c-0bb5-41e8-a858-4b588d3bb4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:57.979966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a9a4c17c-0bb5-41e8-a858-4b588d3bb4f3 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:57.981176 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a9a4c17c-0bb5-41e8-a858-4b588d3bb4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:57.998540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b22393-f8c9-4afd-8648-b41db38d7673 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:58.061893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940c7de-78b3-4017-bd35-939c74f51af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:58.062475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940c7de-78b3-4017-bd35-939c74f51af9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:58.062745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940c7de-78b3-4017-bd35-939c74f51af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:58.063011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940c7de-78b3-4017-bd35-939c74f51af9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:58.079360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940c7de-78b3-4017-bd35-939c74f51af9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:58.081681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940c7de-78b3-4017-bd35-939c74f51af9 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:58.111637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 241/486] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-d940c7de-78b3-4017-bd35-939c74f51af9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:50:58.111931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:58.112860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:58.113880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:50:58.113880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:50:58.113880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:58.113880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:58.113880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 246/487] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:58.118535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4c17c-0bb5-41e8-a858-4b588d3bb4f3 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:58.153713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940c7de-78b3-4017-bd35-939c74f51af9 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:58.220807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:58.269492 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a4c17c-0bb5-41e8-a858-4b588d3bb4f3 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:58.283556 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:58.319493 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940c7de-78b3-4017-bd35-939c74f51af9 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:58.334393 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14282438-94e1-4ba5-a1b2-a2ac5052f299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:58.335374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14282438-94e1-4ba5-a1b2-a2ac5052f299 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:58.335869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14282438-94e1-4ba5-a1b2-a2ac5052f299 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:58.336402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14282438-94e1-4ba5-a1b2-a2ac5052f299 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:58.337471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14282438-94e1-4ba5-a1b2-a2ac5052f299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:58.337781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14282438-94e1-4ba5-a1b2-a2ac5052f299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:58.339183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14282438-94e1-4ba5-a1b2-a2ac5052f299 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:58.367336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 242/488] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:58.372469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14282438-94e1-4ba5-a1b2-a2ac5052f299 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:58.378104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f788e78-79ad-4f48-9c38-29fa53044451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:58.378772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f788e78-79ad-4f48-9c38-29fa53044451 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:58.379051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f788e78-79ad-4f48-9c38-29fa53044451 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:58.379322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f788e78-79ad-4f48-9c38-29fa53044451 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:58.402108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f788e78-79ad-4f48-9c38-29fa53044451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-0f788e78-79ad-4f48-9c38-29fa53044451 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:58.408720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:58.409594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:50:58.410379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:58.411452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 247/489] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:50:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:50:58.420092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f788e78-79ad-4f48-9c38-29fa53044451 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:58.532282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14282438-94e1-4ba5-a1b2-a2ac5052f299 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:58.550705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:58.555351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f788e78-79ad-4f48-9c38-29fa53044451 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:58.581152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1adae11-afc6-4cd6-baa8-430ab0a5f39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:58.581945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adae11-afc6-4cd6-baa8-430ab0a5f39a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:58.582229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adae11-afc6-4cd6-baa8-430ab0a5f39a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:58.582501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adae11-afc6-4cd6-baa8-430ab0a5f39a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:58.592797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1adae11-afc6-4cd6-baa8-430ab0a5f39a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:58.594910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1adae11-afc6-4cd6-baa8-430ab0a5f39a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:58.603949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34618a-c2cc-426d-b473-d0d182584f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:58.604946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34618a-c2cc-426d-b473-d0d182584f2a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:58.605425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34618a-c2cc-426d-b473-d0d182584f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:58.605905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34618a-c2cc-426d-b473-d0d182584f2a None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:58.606822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34618a-c2cc-426d-b473-d0d182584f2a None admin] RBAC: Adding query filter params (name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:58.607339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34618a-c2cc-426d-b473-d0d182584f2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:58.608954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34618a-c2cc-426d-b473-d0d182584f2a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-e1adae11-afc6-4cd6-baa8-430ab0a5f39a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:50:58.609987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:58.611233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:58.612408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:58.612408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:58.612408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:50:58.612408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:50:58.612408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:50:58.612408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:50:58.612408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:50:58.612408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:50:58.612408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:50:58.612408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:58.616846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 243/490] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Aug 1 11:50:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:58.619049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c44a2-86f9-4f97-9e88-c0d44b3ff119 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:58.619338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c44a2-86f9-4f97-9e88-c0d44b3ff119 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:58.619616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c44a2-86f9-4f97-9e88-c0d44b3ff119 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:58.621808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 244/491] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:58.623945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c44a2-86f9-4f97-9e88-c0d44b3ff119 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:58.627949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 248/492] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:50:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:58.629929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205f7e3-800c-4cc5-a0d6-2cb80ee051eb None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:58.630506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205f7e3-800c-4cc5-a0d6-2cb80ee051eb None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:58.630974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205f7e3-800c-4cc5-a0d6-2cb80ee051eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:58.658585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c205f7e3-800c-4cc5-a0d6-2cb80ee051eb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:58.660673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c205f7e3-800c-4cc5-a0d6-2cb80ee051eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:58.674056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:58.757186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 249/493] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:58.758311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205f7e3-800c-4cc5-a0d6-2cb80ee051eb None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:58.777973 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c44a2-86f9-4f97-9e88-c0d44b3ff119 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:58.818908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058afa1-3166-4359-97c5-52bb0081b2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:58.819918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058afa1-3166-4359-97c5-52bb0081b2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:58.820426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058afa1-3166-4359-97c5-52bb0081b2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:58.820897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058afa1-3166-4359-97c5-52bb0081b2f3 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:58.821926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058afa1-3166-4359-97c5-52bb0081b2f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:58.822486 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058afa1-3166-4359-97c5-52bb0081b2f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:58.824160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058afa1-3166-4359-97c5-52bb0081b2f3 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:58.856395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:58.866783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 245/494] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Aug 1 11:50:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:50:58.867092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205f7e3-800c-4cc5-a0d6-2cb80ee051eb None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:58.868134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058afa1-3166-4359-97c5-52bb0081b2f3 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:58.906711 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c06712-d9d9-4fbe-a47b-9485b2390ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:58.907475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c06712-d9d9-4fbe-a47b-9485b2390ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:58.907990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c06712-d9d9-4fbe-a47b-9485b2390ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:58.908109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c06712-d9d9-4fbe-a47b-9485b2390ab5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:58.919593 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c06712-d9d9-4fbe-a47b-9485b2390ab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:58.921871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c06712-d9d9-4fbe-a47b-9485b2390ab5 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-06c06712-d9d9-4fbe-a47b-9485b2390ab5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:50:58.938954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:58.939809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:58.940663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:50:58.940663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:50:58.940663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:58.940663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:58.940663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 250/495] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:58.941354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27abc6d-b7fc-4868-945c-7f7d0710de7c None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:58.941606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27abc6d-b7fc-4868-945c-7f7d0710de7c None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:58.941871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27abc6d-b7fc-4868-945c-7f7d0710de7c None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:58.947192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 251/496] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:58.947980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27abc6d-b7fc-4868-945c-7f7d0710de7c None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:58.974417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:58.987057 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058afa1-3166-4359-97c5-52bb0081b2f3 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:59.027790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e390e-ebda-4a2c-81f4-a0c7e8a845a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:59.028810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e390e-ebda-4a2c-81f4-a0c7e8a845a5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.028931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e390e-ebda-4a2c-81f4-a0c7e8a845a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.029183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e390e-ebda-4a2c-81f4-a0c7e8a845a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.041213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e390e-ebda-4a2c-81f4-a0c7e8a845a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-ec9e390e-ebda-4a2c-81f4-a0c7e8a845a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:59.046555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:59.047437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:59.048244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 246/497] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:50:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:59.048946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e390e-ebda-4a2c-81f4-a0c7e8a845a5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:59.066394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27abc6d-b7fc-4868-945c-7f7d0710de7c None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:59.111250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c385b-a273-44f6-ba8c-19fd24f1bd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:59.111725 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c385b-a273-44f6-ba8c-19fd24f1bd3a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.111960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c385b-a273-44f6-ba8c-19fd24f1bd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.112233 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c385b-a273-44f6-ba8c-19fd24f1bd3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.127152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:59.160027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e390e-ebda-4a2c-81f4-a0c7e8a845a5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:59.185754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c385b-a273-44f6-ba8c-19fd24f1bd3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:59.187219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c385b-a273-44f6-ba8c-19fd24f1bd3a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:59.199419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57a40a-2957-4641-9504-5386029727ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:59.200046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57a40a-2957-4641-9504-5386029727ea None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.200357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57a40a-2957-4641-9504-5386029727ea None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.200628 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57a40a-2957-4641-9504-5386029727ea None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.201332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57a40a-2957-4641-9504-5386029727ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:59.201636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57a40a-2957-4641-9504-5386029727ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:59.203062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57a40a-2957-4641-9504-5386029727ea None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:59.215369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 247/498] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:59.216833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8f840-1f62-4382-9dbf-d6854c687016 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.217134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8f840-1f62-4382-9dbf-d6854c687016 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.217404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8f840-1f62-4382-9dbf-d6854c687016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.236537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-63b8f840-1f62-4382-9dbf-d6854c687016 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:59.237747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-63b8f840-1f62-4382-9dbf-d6854c687016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:59.240623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:59.287002 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 252/499] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:50:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:50:59.288087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c385b-a273-44f6-ba8c-19fd24f1bd3a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:59.306625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 248/500] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:59.308108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8344ca-ed5b-407d-aa2f-7513b221b368 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.308710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8344ca-ed5b-407d-aa2f-7513b221b368 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.308710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8344ca-ed5b-407d-aa2f-7513b221b368 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.310012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 249/501] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:50:59.314391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8344ca-ed5b-407d-aa2f-7513b221b368 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:59.339378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:59.396751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c385b-a273-44f6-ba8c-19fd24f1bd3a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:59.420228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8344ca-ed5b-407d-aa2f-7513b221b368 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:59.431565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29525ed-c99c-4374-8e81-028c7fadc8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:59.432179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29525ed-c99c-4374-8e81-028c7fadc8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.432473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29525ed-c99c-4374-8e81-028c7fadc8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.432740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29525ed-c99c-4374-8e81-028c7fadc8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.433452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29525ed-c99c-4374-8e81-028c7fadc8d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:59.433749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29525ed-c99c-4374-8e81-028c7fadc8d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:59.435127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29525ed-c99c-4374-8e81-028c7fadc8d2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:59.443225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 253/502] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:50:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:59.443998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29525ed-c99c-4374-8e81-028c7fadc8d2 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:59.460019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b1829-79e9-4495-a440-b2a8b865561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:59.460839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b1829-79e9-4495-a440-b2a8b865561b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.461166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b1829-79e9-4495-a440-b2a8b865561b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.461435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b1829-79e9-4495-a440-b2a8b865561b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.468785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b1829-79e9-4495-a440-b2a8b865561b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:59.470239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b1829-79e9-4495-a440-b2a8b865561b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:59.472905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-ec0b1829-79e9-4495-a440-b2a8b865561b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:50:59.482548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:59.483463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:59.484375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 250/503] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:50:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:59.484939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309269ff-caff-476d-bac3-84c157fcd817 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.485475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309269ff-caff-476d-bac3-84c157fcd817 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.485475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309269ff-caff-476d-bac3-84c157fcd817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.508048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-309269ff-caff-476d-bac3-84c157fcd817 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:50:59.509517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-309269ff-caff-476d-bac3-84c157fcd817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:50:59.547685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29525ed-c99c-4374-8e81-028c7fadc8d2 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:59.554734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:59.581182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 251/504] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:50:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:50:59.582295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309269ff-caff-476d-bac3-84c157fcd817 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:59.600986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de916e4-47d8-40ac-ab8b-aaeb4efd300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:59.601650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de916e4-47d8-40ac-ab8b-aaeb4efd300a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.601984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de916e4-47d8-40ac-ab8b-aaeb4efd300a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.602256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de916e4-47d8-40ac-ab8b-aaeb4efd300a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.613672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de916e4-47d8-40ac-ab8b-aaeb4efd300a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:59.615295 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de916e4-47d8-40ac-ab8b-aaeb4efd300a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-4de916e4-47d8-40ac-ab8b-aaeb4efd300a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:50:59.632937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:59.633973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:59.634895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:50:59.634895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:50:59.634895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:59.634895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:59.634895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 254/505] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:59.634895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de916e4-47d8-40ac-ab8b-aaeb4efd300a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:59.687534 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309269ff-caff-476d-bac3-84c157fcd817 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:59.717471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:59.724158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5a487-4c7a-4742-80b2-c1e446a5cf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:59.724847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5a487-4c7a-4742-80b2-c1e446a5cf12 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.725127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5a487-4c7a-4742-80b2-c1e446a5cf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.725395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a5a487-4c7a-4742-80b2-c1e446a5cf12 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.734468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de916e4-47d8-40ac-ab8b-aaeb4efd300a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:59.736249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a5a487-4c7a-4742-80b2-c1e446a5cf12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-20a5a487-4c7a-4742-80b2-c1e446a5cf12 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:50:59.741294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:59.742253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:50:59.743056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 252/506] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:50:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:59.743725 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5a487-4c7a-4742-80b2-c1e446a5cf12 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:59.774388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddb6aa-46fb-4f0d-9443-0ce6267faa5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:59.775029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddb6aa-46fb-4f0d-9443-0ce6267faa5b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.775303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddb6aa-46fb-4f0d-9443-0ce6267faa5b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.775569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddb6aa-46fb-4f0d-9443-0ce6267faa5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.786073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddb6aa-46fb-4f0d-9443-0ce6267faa5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:59.787748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ddb6aa-46fb-4f0d-9443-0ce6267faa5b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-54ddb6aa-46fb-4f0d-9443-0ce6267faa5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:50:59.802028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:59.802913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:59.804017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:50:59.804017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:50:59.804017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:59.804017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:59.804017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 255/507] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:59.804514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddb6aa-46fb-4f0d-9443-0ce6267faa5b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:59.817278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:59.854163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a5a487-4c7a-4742-80b2-c1e446a5cf12 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:59.889903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a1ef7-7699-4a1a-94fe-624306d0f18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:59.890542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a1ef7-7699-4a1a-94fe-624306d0f18e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.890826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a1ef7-7699-4a1a-94fe-624306d0f18e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.891108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a1ef7-7699-4a1a-94fe-624306d0f18e None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.891948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a1ef7-7699-4a1a-94fe-624306d0f18e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:50:59.892258 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a1ef7-7699-4a1a-94fe-624306d0f18e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:59.893791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a1ef7-7699-4a1a-94fe-624306d0f18e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:59.913692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddb6aa-46fb-4f0d-9443-0ce6267faa5b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:50:59.913901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 253/508] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Aug 1 11:50:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:50:59.914676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a1ef7-7699-4a1a-94fe-624306d0f18e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:50:59.954675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e735b-4b8c-4812-a152-c02bca1d6139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:50:59.955330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e735b-4b8c-4812-a152-c02bca1d6139 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:50:59.955706 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e735b-4b8c-4812-a152-c02bca1d6139 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:50:59.956131 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e735b-4b8c-4812-a152-c02bca1d6139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:50:59.970533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:50:59.970897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e735b-4b8c-4812-a152-c02bca1d6139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:50:59.972507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e735b-4b8c-4812-a152-c02bca1d6139 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-287e735b-4b8c-4812-a152-c02bca1d6139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:50:59.988972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:50:59.989962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:50:59.990802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:50:59.990802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:50:59.990802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:50:59.990802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:50:59.991777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 256/509] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:50:59.992569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e735b-4b8c-4812-a152-c02bca1d6139 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.029356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a1ef7-7699-4a1a-94fe-624306d0f18e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.061286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:00.064598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f98882-c67a-44bb-9535-33fddeb23a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:00.065223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98882-c67a-44bb-9535-33fddeb23a3f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:00.065501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98882-c67a-44bb-9535-33fddeb23a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:00.065772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98882-c67a-44bb-9535-33fddeb23a3f None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:00.066476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f98882-c67a-44bb-9535-33fddeb23a3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:00.066773 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f98882-c67a-44bb-9535-33fddeb23a3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:00.068266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f98882-c67a-44bb-9535-33fddeb23a3f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:00.079774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 254/510] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:00.080667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f98882-c67a-44bb-9535-33fddeb23a3f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.108312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e735b-4b8c-4812-a152-c02bca1d6139 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.142905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a091e71-7c59-4fe9-8ca6-41478a4e5e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:00.143533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091e71-7c59-4fe9-8ca6-41478a4e5e62 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:00.143810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091e71-7c59-4fe9-8ca6-41478a4e5e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:00.144086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a091e71-7c59-4fe9-8ca6-41478a4e5e62 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:00.144836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a091e71-7c59-4fe9-8ca6-41478a4e5e62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:00.145140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a091e71-7c59-4fe9-8ca6-41478a4e5e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:00.146629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a091e71-7c59-4fe9-8ca6-41478a4e5e62 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:00.154766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 257/511] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:50:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:00.155573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a091e71-7c59-4fe9-8ca6-41478a4e5e62 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.185006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f98882-c67a-44bb-9535-33fddeb23a3f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.193894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:00.220951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7b6b6-722d-4ade-b020-72833c47aed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:00.221496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7b6b6-722d-4ade-b020-72833c47aed8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:00.221768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7b6b6-722d-4ade-b020-72833c47aed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:00.222033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7b6b6-722d-4ade-b020-72833c47aed8 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:00.222729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7b6b6-722d-4ade-b020-72833c47aed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:00.223028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7b6b6-722d-4ade-b020-72833c47aed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:00.224482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7b6b6-722d-4ade-b020-72833c47aed8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:00.236417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 255/512] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:00.237254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7b6b6-722d-4ade-b020-72833c47aed8 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.260530 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a091e71-7c59-4fe9-8ca6-41478a4e5e62 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.284634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:00.296572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942a023-3638-444f-b71e-fe6266ef4542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:00.297205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942a023-3638-444f-b71e-fe6266ef4542 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:00.297482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942a023-3638-444f-b71e-fe6266ef4542 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:00.297757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942a023-3638-444f-b71e-fe6266ef4542 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:00.337136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7b6b6-722d-4ade-b020-72833c47aed8 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.366103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942a023-3638-444f-b71e-fe6266ef4542 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:00.367581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942a023-3638-444f-b71e-fe6266ef4542 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:00.373217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccb2f5-9888-4e8c-be33-5e55e79c60db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:00.373840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccb2f5-9888-4e8c-be33-5e55e79c60db None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:00.374123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccb2f5-9888-4e8c-be33-5e55e79c60db None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:00.374395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccb2f5-9888-4e8c-be33-5e55e79c60db None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:00.375064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccb2f5-9888-4e8c-be33-5e55e79c60db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:00.375387 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccb2f5-9888-4e8c-be33-5e55e79c60db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:00.376785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccb2f5-9888-4e8c-be33-5e55e79c60db None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:00.388400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 256/513] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:00.389223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccb2f5-9888-4e8c-be33-5e55e79c60db None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.420725 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:00.466885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 258/514] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:51:00.467556 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942a023-3638-444f-b71e-fe6266ef4542 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.491904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:00.495144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccb2f5-9888-4e8c-be33-5e55e79c60db None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.533685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c481e-9802-4ec4-b716-d5bcad409a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:00.533685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c481e-9802-4ec4-b716-d5bcad409a5d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:00.534187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c481e-9802-4ec4-b716-d5bcad409a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:00.534187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c481e-9802-4ec4-b716-d5bcad409a5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:00.541427 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c481e-9802-4ec4-b716-d5bcad409a5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:00.542917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53c481e-9802-4ec4-b716-d5bcad409a5d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-d53c481e-9802-4ec4-b716-d5bcad409a5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:00.555288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:00.556216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:00.557129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:00.557129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:00.557129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:00.557129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:00.557129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:00.557129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:00.557129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:00.557129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:00.557129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 257/515] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:00.557580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c481e-9802-4ec4-b716-d5bcad409a5d None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.577283 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942a023-3638-444f-b71e-fe6266ef4542 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.611726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca697-5948-49cd-a9ea-76d4d4eea201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:00.612381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca697-5948-49cd-a9ea-76d4d4eea201 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:00.612660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca697-5948-49cd-a9ea-76d4d4eea201 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:00.612929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abca697-5948-49cd-a9ea-76d4d4eea201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:00.623034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abca697-5948-49cd-a9ea-76d4d4eea201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:00.624800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abca697-5948-49cd-a9ea-76d4d4eea201 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-6abca697-5948-49cd-a9ea-76d4d4eea201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:00.639197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:00.640065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:00.640958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:00.640958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:00.640958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:00.640958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:00.640958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 259/516] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:00.641229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca697-5948-49cd-a9ea-76d4d4eea201 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.649211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:00.661660 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c481e-9802-4ec4-b716-d5bcad409a5d None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.701591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ee185-8016-4ae0-aca3-78ba036f8533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:00.702236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ee185-8016-4ae0-aca3-78ba036f8533 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:00.702517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ee185-8016-4ae0-aca3-78ba036f8533 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:00.702785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ee185-8016-4ae0-aca3-78ba036f8533 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:00.711052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ee185-8016-4ae0-aca3-78ba036f8533 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:00.713502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489ee185-8016-4ae0-aca3-78ba036f8533 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-489ee185-8016-4ae0-aca3-78ba036f8533 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:00.726521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:00.727468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:00.728377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 258/517] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:00.729002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ee185-8016-4ae0-aca3-78ba036f8533 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.734819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:00.756430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abca697-5948-49cd-a9ea-76d4d4eea201 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.792636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c2ec0-4cf0-4470-a501-2a0a6f049170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:00.793263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c2ec0-4cf0-4470-a501-2a0a6f049170 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:00.793543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c2ec0-4cf0-4470-a501-2a0a6f049170 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:00.793810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c2ec0-4cf0-4470-a501-2a0a6f049170 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:00.794496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c2ec0-4cf0-4470-a501-2a0a6f049170 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:00.795944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c2ec0-4cf0-4470-a501-2a0a6f049170 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:00.819610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 260/518] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:51:00 2022] POST /identity/v3/users => generated 247 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:00.820454 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c2ec0-4cf0-4470-a501-2a0a6f049170 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.843902 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ee185-8016-4ae0-aca3-78ba036f8533 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.884972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c3c31-e524-4b8e-bbc1-3ffe3ef6e243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:00.885624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c3c31-e524-4b8e-bbc1-3ffe3ef6e243 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:00.885909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c3c31-e524-4b8e-bbc1-3ffe3ef6e243 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:00.886187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c3c31-e524-4b8e-bbc1-3ffe3ef6e243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:00.899880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c3c31-e524-4b8e-bbc1-3ffe3ef6e243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:00.901491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c3c31-e524-4b8e-bbc1-3ffe3ef6e243 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:00.911381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-d71c3c31-e524-4b8e-bbc1-3ffe3ef6e243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:00.918971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:00.919904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:00.920787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:00.920787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:00.920787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:00.920787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:00.925098 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 259/519] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:00.925983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c3c31-e524-4b8e-bbc1-3ffe3ef6e243 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:00.959885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c2ec0-4cf0-4470-a501-2a0a6f049170 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:00.998104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:01.008546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974457c9-ea25-44d7-b154-683718803a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:01.009117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974457c9-ea25-44d7-b154-683718803a65 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:01.009381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974457c9-ea25-44d7-b154-683718803a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:01.009965 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974457c9-ea25-44d7-b154-683718803a65 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:01.010401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974457c9-ea25-44d7-b154-683718803a65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:01.010702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974457c9-ea25-44d7-b154-683718803a65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:01.012596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974457c9-ea25-44d7-b154-683718803a65 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:01.036196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 261/520] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:01.037819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974457c9-ea25-44d7-b154-683718803a65 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:01.044414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c3c31-e524-4b8e-bbc1-3ffe3ef6e243 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:01.082156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df9397-7ac9-49a2-9076-c38351c8cb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:01.082822 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df9397-7ac9-49a2-9076-c38351c8cb23 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:01.083106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df9397-7ac9-49a2-9076-c38351c8cb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:01.083455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df9397-7ac9-49a2-9076-c38351c8cb23 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:01.084168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df9397-7ac9-49a2-9076-c38351c8cb23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:01.084506 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df9397-7ac9-49a2-9076-c38351c8cb23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:01.085908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df9397-7ac9-49a2-9076-c38351c8cb23 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:01.101601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 260/521] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:01.102462 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df9397-7ac9-49a2-9076-c38351c8cb23 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:01.154954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974457c9-ea25-44d7-b154-683718803a65 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:01.163267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:01.192794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf2d09-01f3-4061-81ee-fb69a7b3948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:01.193676 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf2d09-01f3-4061-81ee-fb69a7b3948b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:01.194122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf2d09-01f3-4061-81ee-fb69a7b3948b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:01.194654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf2d09-01f3-4061-81ee-fb69a7b3948b None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:01.195465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf2d09-01f3-4061-81ee-fb69a7b3948b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:01.195782 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf2d09-01f3-4061-81ee-fb69a7b3948b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:01.197246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf2d09-01f3-4061-81ee-fb69a7b3948b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:01.206746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df9397-7ac9-49a2-9076-c38351c8cb23 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:01.241048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 262/522] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Aug 1 11:51:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:01.241048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf2d09-01f3-4061-81ee-fb69a7b3948b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:01.250972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f80c63-6753-481b-afbc-18e865041dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:01.251681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f80c63-6753-481b-afbc-18e865041dcd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:01.251881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f80c63-6753-481b-afbc-18e865041dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:01.252154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f80c63-6753-481b-afbc-18e865041dcd None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:01.252897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f80c63-6753-481b-afbc-18e865041dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:01.253203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f80c63-6753-481b-afbc-18e865041dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:01.254589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f80c63-6753-481b-afbc-18e865041dcd None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:01.266549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 261/523] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:01.267251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f80c63-6753-481b-afbc-18e865041dcd None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:01.297418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:01.354246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf2d09-01f3-4061-81ee-fb69a7b3948b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:01.377892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f80c63-6753-481b-afbc-18e865041dcd None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:01.395610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ba949-61d9-4fbd-85aa-945a82a91437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:01.396326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ba949-61d9-4fbd-85aa-945a82a91437 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:01.396594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ba949-61d9-4fbd-85aa-945a82a91437 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:01.396863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ba949-61d9-4fbd-85aa-945a82a91437 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:01.397667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ba949-61d9-4fbd-85aa-945a82a91437 None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=db3ff1b21c784e8cb5bffcb79f230d08) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:01.398085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ba949-61d9-4fbd-85aa-945a82a91437 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:01.399539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ba949-61d9-4fbd-85aa-945a82a91437 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:01.403090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:01.416330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 263/524] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:01 2022] GET /identity/v3/role_assignments?user.id=db3ff1b21c784e8cb5bffcb79f230d08&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:01.416330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ba949-61d9-4fbd-85aa-945a82a91437 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:01.433424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28747e23-2c06-4df8-a852-2b84513cf834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:01.434187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28747e23-2c06-4df8-a852-2b84513cf834 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:01.434533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28747e23-2c06-4df8-a852-2b84513cf834 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:01.434892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28747e23-2c06-4df8-a852-2b84513cf834 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:01.445478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28747e23-2c06-4df8-a852-2b84513cf834 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:01.447377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28747e23-2c06-4df8-a852-2b84513cf834 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-28747e23-2c06-4df8-a852-2b84513cf834 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:01.461382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:01.462349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:01.463240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 262/525] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:01.468545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28747e23-2c06-4df8-a852-2b84513cf834 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:01.538879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ba949-61d9-4fbd-85aa-945a82a91437 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:01.555132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:01.590442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28747e23-2c06-4df8-a852-2b84513cf834 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:01.590840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6116e8-6abc-4da7-b4ba-88b2dd2ef1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:01.591557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6116e8-6abc-4da7-b4ba-88b2dd2ef1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:01.591898 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6116e8-6abc-4da7-b4ba-88b2dd2ef1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:01.592223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6116e8-6abc-4da7-b4ba-88b2dd2ef1b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:01.638606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127079f-2674-483d-944a-be8ceab7c89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:01.639506 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127079f-2674-483d-944a-be8ceab7c89b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:01.640405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127079f-2674-483d-944a-be8ceab7c89b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:01.640841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127079f-2674-483d-944a-be8ceab7c89b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:01.650301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127079f-2674-483d-944a-be8ceab7c89b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:01.652047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127079f-2674-483d-944a-be8ceab7c89b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-2127079f-2674-483d-944a-be8ceab7c89b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:01.668064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:01.669245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:01.670301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:01.670301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:01.670301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:01.670301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:01.670301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:01.670301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:01.670301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:01.670301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:01.670800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6116e8-6abc-4da7-b4ba-88b2dd2ef1b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:01.672224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 263/526] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:01.673167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127079f-2674-483d-944a-be8ceab7c89b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:01.677141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6116e8-6abc-4da7-b4ba-88b2dd2ef1b7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:01.729006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:01.784720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 264/527] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:51:01.784720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6116e8-6abc-4da7-b4ba-88b2dd2ef1b7 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:01.803332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:01.815326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127079f-2674-483d-944a-be8ceab7c89b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:01.860250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e42952-6c7b-47f0-a89f-24a82e8a3e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:01.861622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e42952-6c7b-47f0-a89f-24a82e8a3e2a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:01.862234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e42952-6c7b-47f0-a89f-24a82e8a3e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:01.862799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e42952-6c7b-47f0-a89f-24a82e8a3e2a None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:01.863897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e42952-6c7b-47f0-a89f-24a82e8a3e2a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:01.864455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e42952-6c7b-47f0-a89f-24a82e8a3e2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:01.866135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e42952-6c7b-47f0-a89f-24a82e8a3e2a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:01.885639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 264/528] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Aug 1 11:51:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:01.886888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e42952-6c7b-47f0-a89f-24a82e8a3e2a None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:01.900650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6116e8-6abc-4da7-b4ba-88b2dd2ef1b7 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:01.938642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c631f6-8300-468a-9af8-344bb405f17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:01.939325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c631f6-8300-468a-9af8-344bb405f17a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:01.939837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c631f6-8300-468a-9af8-344bb405f17a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:01.940125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c631f6-8300-468a-9af8-344bb405f17a None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:01.941003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c631f6-8300-468a-9af8-344bb405f17a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:01.941680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c631f6-8300-468a-9af8-344bb405f17a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:01.943214 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c631f6-8300-468a-9af8-344bb405f17a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:01.964333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 265/529] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:01.964333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c631f6-8300-468a-9af8-344bb405f17a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:02.003802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:02.009617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e42952-6c7b-47f0-a89f-24a82e8a3e2a None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:02.066937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:02.067256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9537bd1-8754-474a-a01d-bf67be539902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:02.068422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9537bd1-8754-474a-a01d-bf67be539902 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:02.068422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9537bd1-8754-474a-a01d-bf67be539902 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:02.069074 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9537bd1-8754-474a-a01d-bf67be539902 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:02.080448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9537bd1-8754-474a-a01d-bf67be539902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:02.082044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9537bd1-8754-474a-a01d-bf67be539902 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:02.095758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c631f6-8300-468a-9af8-344bb405f17a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-a9537bd1-8754-474a-a01d-bf67be539902 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:02.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:02.100147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:02.101192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:02.101192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:02.101192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:02.101192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:02.106672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 265/530] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:02.107679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9537bd1-8754-474a-a01d-bf67be539902 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:02.134471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696915a-7b2e-4485-80fb-d1700ee57d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:02.135326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696915a-7b2e-4485-80fb-d1700ee57d42 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:02.135741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696915a-7b2e-4485-80fb-d1700ee57d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:02.136159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696915a-7b2e-4485-80fb-d1700ee57d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:02.205184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696915a-7b2e-4485-80fb-d1700ee57d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:02.206904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696915a-7b2e-4485-80fb-d1700ee57d42 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:02.214507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9537bd1-8754-474a-a01d-bf67be539902 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:02.251622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed55c6-da55-4e03-b9d8-365f0d6696eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:02.252468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed55c6-da55-4e03-b9d8-365f0d6696eb None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:02.252845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed55c6-da55-4e03-b9d8-365f0d6696eb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:02.253222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed55c6-da55-4e03-b9d8-365f0d6696eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:02.254112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed55c6-da55-4e03-b9d8-365f0d6696eb None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=184005a383ff4ab696d5235369b44628) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:02.254519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed55c6-da55-4e03-b9d8-365f0d6696eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:02.256125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed55c6-da55-4e03-b9d8-365f0d6696eb None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:02.268466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 266/531] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:02 2022] GET /identity/v3/role_assignments?user.id=184005a383ff4ab696d5235369b44628&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:02.268466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed55c6-da55-4e03-b9d8-365f0d6696eb None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:02.282153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:02.307605 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:02.363716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 266/532] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:51:02.369722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696915a-7b2e-4485-80fb-d1700ee57d42 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:02.380505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed55c6-da55-4e03-b9d8-365f0d6696eb None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:02.424632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed36731-7655-45a8-a039-469c16dd8eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:02.425646 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed36731-7655-45a8-a039-469c16dd8eec None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:02.426155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed36731-7655-45a8-a039-469c16dd8eec None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:02.426802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed36731-7655-45a8-a039-469c16dd8eec None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:02.427824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed36731-7655-45a8-a039-469c16dd8eec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:02.428385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed36731-7655-45a8-a039-469c16dd8eec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:02.430253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed36731-7655-45a8-a039-469c16dd8eec None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:02.444717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 267/533] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:02.451131 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed36731-7655-45a8-a039-469c16dd8eec None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:02.538127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:02.566009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed36731-7655-45a8-a039-469c16dd8eec None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:02.572342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696915a-7b2e-4485-80fb-d1700ee57d42 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:02.602910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287e38d-ce99-4ca6-bd92-7189c97ff854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:02.603957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287e38d-ce99-4ca6-bd92-7189c97ff854 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:02.604486 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287e38d-ce99-4ca6-bd92-7189c97ff854 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:02.604969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287e38d-ce99-4ca6-bd92-7189c97ff854 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:02.612823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287e38d-ce99-4ca6-bd92-7189c97ff854 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:02.614687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287e38d-ce99-4ca6-bd92-7189c97ff854 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-5287e38d-ce99-4ca6-bd92-7189c97ff854 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:02.626980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:02.627884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:02.628781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:02.628781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:02.628781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:02.628781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:02.628781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:02.628781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:02.628781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:02.628781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:02.631412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 268/534] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:02.637261 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287e38d-ce99-4ca6-bd92-7189c97ff854 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:02.639191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9fbd6-07f8-4e6b-b4f7-96b4ee7bb80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:02.639871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9fbd6-07f8-4e6b-b4f7-96b4ee7bb80c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:02.640151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9fbd6-07f8-4e6b-b4f7-96b4ee7bb80c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:02.640453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d9fbd6-07f8-4e6b-b4f7-96b4ee7bb80c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:02.662460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9fbd6-07f8-4e6b-b4f7-96b4ee7bb80c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:02.663941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d9fbd6-07f8-4e6b-b4f7-96b4ee7bb80c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-81d9fbd6-07f8-4e6b-b4f7-96b4ee7bb80c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:02.694078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:02.695023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:02.695837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:02.695837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:02.695837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:02.695837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:02.704891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 267/535] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:02.705977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9fbd6-07f8-4e6b-b4f7-96b4ee7bb80c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:02.749812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:02.754404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287e38d-ce99-4ca6-bd92-7189c97ff854 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:02.846340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d9fbd6-07f8-4e6b-b4f7-96b4ee7bb80c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:02.847351 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff752d02-ec3c-4d02-888e-6a541a738527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:02.848001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff752d02-ec3c-4d02-888e-6a541a738527 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:02.848283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff752d02-ec3c-4d02-888e-6a541a738527 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:02.848579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff752d02-ec3c-4d02-888e-6a541a738527 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:02.849461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff752d02-ec3c-4d02-888e-6a541a738527 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:02.849755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff752d02-ec3c-4d02-888e-6a541a738527 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:02.851223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff752d02-ec3c-4d02-888e-6a541a738527 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:02.854289 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:02.890494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 269/536] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:02.891831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9939e-4e45-43d2-9f45-d7eec9421674 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:02.892123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9939e-4e45-43d2-9f45-d7eec9421674 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:02.892841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5378955-56f0-4861-8482-e82610559e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:02.893808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5378955-56f0-4861-8482-e82610559e5c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:02.894294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5378955-56f0-4861-8482-e82610559e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:02.894875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5378955-56f0-4861-8482-e82610559e5c None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:02.895834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5378955-56f0-4861-8482-e82610559e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:02.896243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5378955-56f0-4861-8482-e82610559e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:02.896538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9939e-4e45-43d2-9f45-d7eec9421674 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:02.897859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5378955-56f0-4861-8482-e82610559e5c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:02.898314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 270/537] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:02.899197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9939e-4e45-43d2-9f45-d7eec9421674 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:02.916381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 268/538] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:02.917925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a241eb-3f27-4c40-aba2-f212aa9d84f9 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:02.918382 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a241eb-3f27-4c40-aba2-f212aa9d84f9 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:02.918793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a241eb-3f27-4c40-aba2-f212aa9d84f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:02.937740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c4a241eb-3f27-4c40-aba2-f212aa9d84f9 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:02.939222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c4a241eb-3f27-4c40-aba2-f212aa9d84f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:03.011185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 269/539] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:03.012614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a241eb-3f27-4c40-aba2-f212aa9d84f9 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:03.034578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:03.036390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9939e-4e45-43d2-9f45-d7eec9421674 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:03.084837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225d4e4-a7fe-4fb7-8166-cbb86bfa84ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:03.085899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225d4e4-a7fe-4fb7-8166-cbb86bfa84ee None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:03.086283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225d4e4-a7fe-4fb7-8166-cbb86bfa84ee None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:03.087135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225d4e4-a7fe-4fb7-8166-cbb86bfa84ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:03.088352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225d4e4-a7fe-4fb7-8166-cbb86bfa84ee None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=dc0102a256934641a999dd091e6ba0b3) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:03.088772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225d4e4-a7fe-4fb7-8166-cbb86bfa84ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:03.091016 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225d4e4-a7fe-4fb7-8166-cbb86bfa84ee None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:03.104347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 271/540] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:02 2022] GET /identity/v3/role_assignments?user.id=dc0102a256934641a999dd091e6ba0b3&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:03.104347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225d4e4-a7fe-4fb7-8166-cbb86bfa84ee None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:03.172835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a241eb-3f27-4c40-aba2-f212aa9d84f9 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:03.216456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b18a1-b4b9-4829-9edf-a88091d6f89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:03.217016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b18a1-b4b9-4829-9edf-a88091d6f89b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:03.217292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b18a1-b4b9-4829-9edf-a88091d6f89b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:03.217564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853b18a1-b4b9-4829-9edf-a88091d6f89b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:03.223495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:03.227373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b18a1-b4b9-4829-9edf-a88091d6f89b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:03.230118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853b18a1-b4b9-4829-9edf-a88091d6f89b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-853b18a1-b4b9-4829-9edf-a88091d6f89b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:03.244476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:03.245394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:03.246232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:03.246232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:03.246232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:03.246232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:03.246232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:03.246232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:03.246232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:03.246232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:03.253315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 270/541] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:03.255066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b18a1-b4b9-4829-9edf-a88091d6f89b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:03.327124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225d4e4-a7fe-4fb7-8166-cbb86bfa84ee None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:03.341811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:03.369753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853b18a1-b4b9-4829-9edf-a88091d6f89b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:03.386650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641797-b343-4f5b-8020-dfb7474c753e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:03.387356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641797-b343-4f5b-8020-dfb7474c753e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:03.387622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641797-b343-4f5b-8020-dfb7474c753e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:03.387943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c641797-b343-4f5b-8020-dfb7474c753e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:03.402288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c641797-b343-4f5b-8020-dfb7474c753e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-7c641797-b343-4f5b-8020-dfb7474c753e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:03.408524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:03.409412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:03.410631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 272/542] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:03.414701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb32c8-bc38-4512-baa9-1cb81ca60e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:03.415576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb32c8-bc38-4512-baa9-1cb81ca60e17 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:03.415680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641797-b343-4f5b-8020-dfb7474c753e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:03.416272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb32c8-bc38-4512-baa9-1cb81ca60e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:03.416787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb32c8-bc38-4512-baa9-1cb81ca60e17 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:03.417746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb32c8-bc38-4512-baa9-1cb81ca60e17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:03.418358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb32c8-bc38-4512-baa9-1cb81ca60e17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:03.420109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb32c8-bc38-4512-baa9-1cb81ca60e17 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:03.439708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 271/543] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:03.441823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb32c8-bc38-4512-baa9-1cb81ca60e17 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:03.561276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb32c8-bc38-4512-baa9-1cb81ca60e17 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:03.565137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c641797-b343-4f5b-8020-dfb7474c753e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:03.596456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:03.600846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1645e-63c2-496a-bbc8-c931babd6769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:03.602674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1645e-63c2-496a-bbc8-c931babd6769 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:03.603162 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1645e-63c2-496a-bbc8-c931babd6769 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:03.603626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1645e-63c2-496a-bbc8-c931babd6769 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:03.604689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1645e-63c2-496a-bbc8-c931babd6769 None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=8dfbda2838b242558c885f913fafb5d8) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:03.605386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1645e-63c2-496a-bbc8-c931babd6769 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:03.607254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1645e-63c2-496a-bbc8-c931babd6769 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:03.611341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:03.620490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 272/544] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:03 2022] GET /identity/v3/role_assignments?user.id=8dfbda2838b242558c885f913fafb5d8&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:03.631750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6ab8a5-1c1b-447e-a594-d94ccf53c900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:03.632447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ab8a5-1c1b-447e-a594-d94ccf53c900 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:03.632731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ab8a5-1c1b-447e-a594-d94ccf53c900 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:03.633004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6ab8a5-1c1b-447e-a594-d94ccf53c900 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:03.633756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ab8a5-1c1b-447e-a594-d94ccf53c900 None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:03.634060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ab8a5-1c1b-447e-a594-d94ccf53c900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:03.635614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6ab8a5-1c1b-447e-a594-d94ccf53c900 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:03.646209 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 273/545] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:03.651640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1645e-63c2-496a-bbc8-c931babd6769 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:03.703659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345007c-5272-4e7f-b20a-2d3b60207512 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:03.703964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345007c-5272-4e7f-b20a-2d3b60207512 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:03.704146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345007c-5272-4e7f-b20a-2d3b60207512 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:03.705898 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 274/546] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:03.718460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03a464-4181-4e7a-bc75-e53eae2b3a45 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:03.718781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03a464-4181-4e7a-bc75-e53eae2b3a45 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:03.719014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03a464-4181-4e7a-bc75-e53eae2b3a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:03.759168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-dd03a464-4181-4e7a-bc75-e53eae2b3a45 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:03.760765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-dd03a464-4181-4e7a-bc75-e53eae2b3a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:03.788703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1645e-63c2-496a-bbc8-c931babd6769 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:03.827957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4633478-8dee-4497-8571-baad50ca8107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:03.829001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4633478-8dee-4497-8571-baad50ca8107 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:03.829504 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4633478-8dee-4497-8571-baad50ca8107 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:03.830048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4633478-8dee-4497-8571-baad50ca8107 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:03.849263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4633478-8dee-4497-8571-baad50ca8107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:03.851028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4633478-8dee-4497-8571-baad50ca8107 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-a4633478-8dee-4497-8571-baad50ca8107 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:03.867453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:03.868496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:03.869768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:03.869768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:03.869768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:03.869768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:03.869768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 273/547] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:03.874439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4633478-8dee-4497-8571-baad50ca8107 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:03.894481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 275/548] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:03.941327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03a464-4181-4e7a-bc75-e53eae2b3a45 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:03.943509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:04.043257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4633478-8dee-4497-8571-baad50ca8107 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:04.080626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666c47a-e6f0-41a8-b06f-9b4816dd5092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:04.081262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c47a-e6f0-41a8-b06f-9b4816dd5092 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:04.081538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c47a-e6f0-41a8-b06f-9b4816dd5092 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:04.081812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666c47a-e6f0-41a8-b06f-9b4816dd5092 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:04.082514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666c47a-e6f0-41a8-b06f-9b4816dd5092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:04.082818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666c47a-e6f0-41a8-b06f-9b4816dd5092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:04.084322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666c47a-e6f0-41a8-b06f-9b4816dd5092 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:04.089930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03a464-4181-4e7a-bc75-e53eae2b3a45 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:04.111275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:04.116969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 274/549] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:04.120785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666c47a-e6f0-41a8-b06f-9b4816dd5092 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:04.173998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fa047-db3b-43c7-b665-d8dd7139e8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:04.175032 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fa047-db3b-43c7-b665-d8dd7139e8be None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:04.175539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fa047-db3b-43c7-b665-d8dd7139e8be None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:04.176006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fa047-db3b-43c7-b665-d8dd7139e8be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:04.190930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54fa047-db3b-43c7-b665-d8dd7139e8be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-d54fa047-db3b-43c7-b665-d8dd7139e8be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:04.196537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:04.197948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:04.198915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:04.202004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 276/550] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:04.208081 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fa047-db3b-43c7-b665-d8dd7139e8be None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:04.280331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666c47a-e6f0-41a8-b06f-9b4816dd5092 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:04.297081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:04.319425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c88f1-8176-4549-844b-2c3d9ed72b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:04.320530 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c88f1-8176-4549-844b-2c3d9ed72b82 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:04.321058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c88f1-8176-4549-844b-2c3d9ed72b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:04.321526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c88f1-8176-4549-844b-2c3d9ed72b82 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:04.329581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c88f1-8176-4549-844b-2c3d9ed72b82 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:04.331610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c88f1-8176-4549-844b-2c3d9ed72b82 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-879c88f1-8176-4549-844b-2c3d9ed72b82 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:04.363682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:04.364738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:04.365727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:04.365727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:04.365727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:04.365727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:04.365727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:04.365727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:04.365727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:04.365727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:04.365727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:04.365727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:04.373538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 275/551] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:04.374466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c88f1-8176-4549-844b-2c3d9ed72b82 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:04.396495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54fa047-db3b-43c7-b665-d8dd7139e8be None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:04.469497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08809144-292b-46bf-89aa-8c3d33dd1f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:04.470133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08809144-292b-46bf-89aa-8c3d33dd1f65 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:04.470410 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08809144-292b-46bf-89aa-8c3d33dd1f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:04.470681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08809144-292b-46bf-89aa-8c3d33dd1f65 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:04.471414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08809144-292b-46bf-89aa-8c3d33dd1f65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:04.471719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08809144-292b-46bf-89aa-8c3d33dd1f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:04.473155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08809144-292b-46bf-89aa-8c3d33dd1f65 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:04.495690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 277/552] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:04.518118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08809144-292b-46bf-89aa-8c3d33dd1f65 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:04.564726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c88f1-8176-4549-844b-2c3d9ed72b82 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:04.568012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:04.603171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a76997-938b-4f20-a86d-a0b65301065c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:04.603853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a76997-938b-4f20-a86d-a0b65301065c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:04.608327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a76997-938b-4f20-a86d-a0b65301065c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:04.608327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a76997-938b-4f20-a86d-a0b65301065c None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:04.608327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a76997-938b-4f20-a86d-a0b65301065c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:04.608327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a76997-938b-4f20-a86d-a0b65301065c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:04.608327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a76997-938b-4f20-a86d-a0b65301065c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:04.635143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 276/553] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Aug 1 11:51:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:04.636916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf5a10-7e35-4e7f-9f19-80a95b6ff2bb None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:04.637209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf5a10-7e35-4e7f-9f19-80a95b6ff2bb None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:04.637487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf5a10-7e35-4e7f-9f19-80a95b6ff2bb None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:04.638952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 277/554] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:04.641215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf5a10-7e35-4e7f-9f19-80a95b6ff2bb None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:04.650677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08809144-292b-46bf-89aa-8c3d33dd1f65 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:04.685440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:04.698384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721b535-bf01-421a-b45f-e29fa9d71f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:04.699912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721b535-bf01-421a-b45f-e29fa9d71f20 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:04.700382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721b535-bf01-421a-b45f-e29fa9d71f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:04.701358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721b535-bf01-421a-b45f-e29fa9d71f20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:04.716720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721b535-bf01-421a-b45f-e29fa9d71f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:04.719338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721b535-bf01-421a-b45f-e29fa9d71f20 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-5721b535-bf01-421a-b45f-e29fa9d71f20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:04.741962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:04.743124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:04.744208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:04.744208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:04.744208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:04.744208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:04.751578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 278/555] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:04.751578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd4e8e-0b68-4944-ae1e-4c0692fb67f1 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:04.751578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd4e8e-0b68-4944-ae1e-4c0692fb67f1 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:04.751578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd4e8e-0b68-4944-ae1e-4c0692fb67f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:04.772929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf5a10-7e35-4e7f-9f19-80a95b6ff2bb None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:04.776365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-60dd4e8e-0b68-4944-ae1e-4c0692fb67f1 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:04.778736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-60dd4e8e-0b68-4944-ae1e-4c0692fb67f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:04.810731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9542e-03b0-49d6-b518-e04ec47c04df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:04.811708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9542e-03b0-49d6-b518-e04ec47c04df None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:04.811708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9542e-03b0-49d6-b518-e04ec47c04df None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:04.812049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9542e-03b0-49d6-b518-e04ec47c04df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:04.874276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 279/556] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:04.875462 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd4e8e-0b68-4944-ae1e-4c0692fb67f1 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:04.892477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:04.903009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9542e-03b0-49d6-b518-e04ec47c04df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:04.905387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9542e-03b0-49d6-b518-e04ec47c04df None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:04.943764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:04.983522 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd4e8e-0b68-4944-ae1e-4c0692fb67f1 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:05.028471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 278/557] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 11:51:05.029796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9542e-03b0-49d6-b518-e04ec47c04df None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:05.030828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ebe72f-a174-4e83-b8fe-1d8523300040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:05.031674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebe72f-a174-4e83-b8fe-1d8523300040 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.032370 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebe72f-a174-4e83-b8fe-1d8523300040 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.032903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ebe72f-a174-4e83-b8fe-1d8523300040 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.034736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ebe72f-a174-4e83-b8fe-1d8523300040 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:05.035255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ebe72f-a174-4e83-b8fe-1d8523300040 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:05.036944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ebe72f-a174-4e83-b8fe-1d8523300040 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:05.057553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 280/558] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:05.057553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ebe72f-a174-4e83-b8fe-1d8523300040 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:05.139830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:05.160823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9542e-03b0-49d6-b518-e04ec47c04df None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:05.169111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ebe72f-a174-4e83-b8fe-1d8523300040 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:05.203603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:05.209186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22afc8d-3641-4a7a-95c3-3a764fca1ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:05.210037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22afc8d-3641-4a7a-95c3-3a764fca1ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.210414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22afc8d-3641-4a7a-95c3-3a764fca1ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.210792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22afc8d-3641-4a7a-95c3-3a764fca1ad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.211823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713f0b2-af07-4bc6-8c40-a48b4a55803c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:05.213371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713f0b2-af07-4bc6-8c40-a48b4a55803c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.213891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713f0b2-af07-4bc6-8c40-a48b4a55803c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.214790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713f0b2-af07-4bc6-8c40-a48b4a55803c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.225043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22afc8d-3641-4a7a-95c3-3a764fca1ad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:05.226653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22afc8d-3641-4a7a-95c3-3a764fca1ad3 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:05.227789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713f0b2-af07-4bc6-8c40-a48b4a55803c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-e713f0b2-af07-4bc6-8c40-a48b4a55803c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:05.234979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:05.236534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:05.237375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:05.240467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 279/559] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:05.241501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713f0b2-af07-4bc6-8c40-a48b4a55803c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-c22afc8d-3641-4a7a-95c3-3a764fca1ad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:05.245669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:05.246626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:05.247474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:05.247474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:05.247474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:05.247474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:05.247474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 281/560] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:05.248231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22afc8d-3641-4a7a-95c3-3a764fca1ad3 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:05.358297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713f0b2-af07-4bc6-8c40-a48b4a55803c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:05.361984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22afc8d-3641-4a7a-95c3-3a764fca1ad3 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:05.388697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:05.393677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254db7c-cc84-4546-87da-6bb3f5fc1215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:05.394277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254db7c-cc84-4546-87da-6bb3f5fc1215 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.394796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254db7c-cc84-4546-87da-6bb3f5fc1215 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.395118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254db7c-cc84-4546-87da-6bb3f5fc1215 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.401840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ed2f7-ad1c-4a70-a78c-38f5ff65ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:05.403140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ed2f7-ad1c-4a70-a78c-38f5ff65ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.403628 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ed2f7-ad1c-4a70-a78c-38f5ff65ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.404111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ed2f7-ad1c-4a70-a78c-38f5ff65ddc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.405090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ed2f7-ad1c-4a70-a78c-38f5ff65ddc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:05.405434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254db7c-cc84-4546-87da-6bb3f5fc1215 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:05.405823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ed2f7-ad1c-4a70-a78c-38f5ff65ddc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:05.406911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254db7c-cc84-4546-87da-6bb3f5fc1215 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:05.407651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ed2f7-ad1c-4a70-a78c-38f5ff65ddc7 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:05.417067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 282/561] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:05.418192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ed2f7-ad1c-4a70-a78c-38f5ff65ddc7 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-c254db7c-cc84-4546-87da-6bb3f5fc1215 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:05.420237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:05.421227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:05.422243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:05.422243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:05.422243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:05.422243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:05.422243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:05.422243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:05.422243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:05.422243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:05.422243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:05.422243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:05.424364 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 280/562] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:05.425412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254db7c-cc84-4546-87da-6bb3f5fc1215 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:05.498978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:05.538459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ed2f7-ad1c-4a70-a78c-38f5ff65ddc7 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:05.563199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254db7c-cc84-4546-87da-6bb3f5fc1215 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:05.572436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf531520-93f8-442c-aca4-870b9da88c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:05.573028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf531520-93f8-442c-aca4-870b9da88c12 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.573564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf531520-93f8-442c-aca4-870b9da88c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.573564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf531520-93f8-442c-aca4-870b9da88c12 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.574304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf531520-93f8-442c-aca4-870b9da88c12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:05.574586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf531520-93f8-442c-aca4-870b9da88c12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:05.576042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf531520-93f8-442c-aca4-870b9da88c12 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:05.592685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 283/563] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:05.592685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf531520-93f8-442c-aca4-870b9da88c12 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:05.603669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ebc798-7dd6-41a8-99fe-5f3577b544fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:05.604625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebc798-7dd6-41a8-99fe-5f3577b544fb None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.604625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebc798-7dd6-41a8-99fe-5f3577b544fb None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.604921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ebc798-7dd6-41a8-99fe-5f3577b544fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.616609 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebc798-7dd6-41a8-99fe-5f3577b544fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:05.618165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ebc798-7dd6-41a8-99fe-5f3577b544fb None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-b0ebc798-7dd6-41a8-99fe-5f3577b544fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:05.632405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:05.633337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:05.638989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:05.638989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:05.638989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:05.638989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:05.638989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 281/564] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:05.638989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b5b8-57f1-4dc5-8ac3-ccd5e1e9ff24 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.638989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b5b8-57f1-4dc5-8ac3-ccd5e1e9ff24 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.638989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97b5b8-57f1-4dc5-8ac3-ccd5e1e9ff24 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.638989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 282/565] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:05.639514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97b5b8-57f1-4dc5-8ac3-ccd5e1e9ff24 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:05.650751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:05.705716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf531520-93f8-442c-aca4-870b9da88c12 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:05.743758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8abbc-ed75-4887-8259-d41327f6bc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:05.744491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8abbc-ed75-4887-8259-d41327f6bc07 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.744776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8abbc-ed75-4887-8259-d41327f6bc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.745054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8abbc-ed75-4887-8259-d41327f6bc07 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.745938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8abbc-ed75-4887-8259-d41327f6bc07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:05.746249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8abbc-ed75-4887-8259-d41327f6bc07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:05.748250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de8abbc-ed75-4887-8259-d41327f6bc07 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:05.752739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97b5b8-57f1-4dc5-8ac3-ccd5e1e9ff24 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:05.754737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:05.769122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 284/566] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:05.769845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8abbc-ed75-4887-8259-d41327f6bc07 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:05.802909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5347e-1024-4f96-9e08-efa56e71bb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:05.804023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5347e-1024-4f96-9e08-efa56e71bb8e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.804489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5347e-1024-4f96-9e08-efa56e71bb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.804905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5347e-1024-4f96-9e08-efa56e71bb8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.813765 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5347e-1024-4f96-9e08-efa56e71bb8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:05.815226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5347e-1024-4f96-9e08-efa56e71bb8e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-69d5347e-1024-4f96-9e08-efa56e71bb8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:05.827576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:05.828526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:05.829834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:05.829834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:05.829834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:05.829834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:05.829834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:05.829834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:05.829834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:05.829834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:05.829834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 283/567] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:05.830546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda09e36-8538-47fc-89ee-15d9b801b255 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.830617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda09e36-8538-47fc-89ee-15d9b801b255 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.830880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda09e36-8538-47fc-89ee-15d9b801b255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.853635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-fda09e36-8538-47fc-89ee-15d9b801b255 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:05.855569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fda09e36-8538-47fc-89ee-15d9b801b255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:05.892616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8abbc-ed75-4887-8259-d41327f6bc07 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:05.900591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:05.928362 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c04fd3-ecc3-4d07-be76-e3a184bf8b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:05.929008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c04fd3-ecc3-4d07-be76-e3a184bf8b49 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:05.929286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c04fd3-ecc3-4d07-be76-e3a184bf8b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:05.929558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c04fd3-ecc3-4d07-be76-e3a184bf8b49 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:05.930257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c04fd3-ecc3-4d07-be76-e3a184bf8b49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:05.930562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c04fd3-ecc3-4d07-be76-e3a184bf8b49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:05.931941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c04fd3-ecc3-4d07-be76-e3a184bf8b49 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:05.932430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 284/568] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:05.933701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda09e36-8538-47fc-89ee-15d9b801b255 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:05.944328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 285/569] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:05.949466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c04fd3-ecc3-4d07-be76-e3a184bf8b49 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.023667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:06.033321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda09e36-8538-47fc-89ee-15d9b801b255 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.047836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c04fd3-ecc3-4d07-be76-e3a184bf8b49 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.069143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf785a2-4022-48e1-afb5-e352518d0127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.069814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf785a2-4022-48e1-afb5-e352518d0127 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.070089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf785a2-4022-48e1-afb5-e352518d0127 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.070360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf785a2-4022-48e1-afb5-e352518d0127 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.080498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf785a2-4022-48e1-afb5-e352518d0127 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:06.080670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261235b0-feb0-431d-8ded-3a435b78ed12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.081272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261235b0-feb0-431d-8ded-3a435b78ed12 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.081547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261235b0-feb0-431d-8ded-3a435b78ed12 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.081816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261235b0-feb0-431d-8ded-3a435b78ed12 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.081962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf785a2-4022-48e1-afb5-e352518d0127 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:06.082562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261235b0-feb0-431d-8ded-3a435b78ed12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:06.082873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261235b0-feb0-431d-8ded-3a435b78ed12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:06.084436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261235b0-feb0-431d-8ded-3a435b78ed12 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-7bf785a2-4022-48e1-afb5-e352518d0127 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:06.097578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:06.098548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:06.099432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:06.099432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:06.099432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:06.099432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:06.099432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 285/570] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:06.100091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 286/571] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:06.100617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e08add-6bed-4c0b-a5ec-33ec0487f373 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.100932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261235b0-feb0-431d-8ded-3a435b78ed12 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.101460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e08add-6bed-4c0b-a5ec-33ec0487f373 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.101955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e08add-6bed-4c0b-a5ec-33ec0487f373 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.103773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 286/572] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:06.104507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e08add-6bed-4c0b-a5ec-33ec0487f373 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.131786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:06.205115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261235b0-feb0-431d-8ded-3a435b78ed12 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.209177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e08add-6bed-4c0b-a5ec-33ec0487f373 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.245702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385b4aa-c7cc-48b6-8008-c0a1df327cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.246361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385b4aa-c7cc-48b6-8008-c0a1df327cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.246641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385b4aa-c7cc-48b6-8008-c0a1df327cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.246920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385b4aa-c7cc-48b6-8008-c0a1df327cf8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.247705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deaa5c-4415-480a-9346-04aff18ab547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.248243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deaa5c-4415-480a-9346-04aff18ab547 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.248541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deaa5c-4415-480a-9346-04aff18ab547 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.248796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7deaa5c-4415-480a-9346-04aff18ab547 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.254446 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385b4aa-c7cc-48b6-8008-c0a1df327cf8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:06.255944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385b4aa-c7cc-48b6-8008-c0a1df327cf8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:06.258871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7deaa5c-4415-480a-9346-04aff18ab547 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-a7deaa5c-4415-480a-9346-04aff18ab547 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:06.264163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:06.265164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:06.266005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 287/573] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:06.266681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deaa5c-4415-480a-9346-04aff18ab547 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-8385b4aa-c7cc-48b6-8008-c0a1df327cf8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:06.267713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:06.268867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 287/574] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:06.270146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385b4aa-c7cc-48b6-8008-c0a1df327cf8 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.278230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:06.363403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:06.380020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7deaa5c-4415-480a-9346-04aff18ab547 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.380245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385b4aa-c7cc-48b6-8008-c0a1df327cf8 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.417495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7f037-5ec0-4b32-a401-3b46e4814788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.418056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb67e8-7d33-455c-971d-051861ecc5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.418518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f037-5ec0-4b32-a401-3b46e4814788 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.418518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f037-5ec0-4b32-a401-3b46e4814788 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.418715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f037-5ec0-4b32-a401-3b46e4814788 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.418798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb67e8-7d33-455c-971d-051861ecc5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.418944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb67e8-7d33-455c-971d-051861ecc5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.419228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb67e8-7d33-455c-971d-051861ecc5c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.419331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7f037-5ec0-4b32-a401-3b46e4814788 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:06.419633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7f037-5ec0-4b32-a401-3b46e4814788 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:06.420029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb67e8-7d33-455c-971d-051861ecc5c3 None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=b9b6a96421e349a88a38590cfce167fb) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:06.420368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb67e8-7d33-455c-971d-051861ecc5c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:06.421058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7f037-5ec0-4b32-a401-3b46e4814788 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:06.421797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fb67e8-7d33-455c-971d-051861ecc5c3 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:06.427632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 288/575] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:06 2022] GET /identity/v3/role_assignments?user.id=b9b6a96421e349a88a38590cfce167fb&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:06.428950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620c536-d1a7-4e2d-b6fa-9fa1bb73cde6 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.429236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620c536-d1a7-4e2d-b6fa-9fa1bb73cde6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.429516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620c536-d1a7-4e2d-b6fa-9fa1bb73cde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.437831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 288/576] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:06.437831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7f037-5ec0-4b32-a401-3b46e4814788 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.456075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a620c536-d1a7-4e2d-b6fa-9fa1bb73cde6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:06.457294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a620c536-d1a7-4e2d-b6fa-9fa1bb73cde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:06.509129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:06.535554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 289/577] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:06.536696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620c536-d1a7-4e2d-b6fa-9fa1bb73cde6 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.564445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7f037-5ec0-4b32-a401-3b46e4814788 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.608603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e6945-f2df-4172-bdef-eec5cc386e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.609404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e6945-f2df-4172-bdef-eec5cc386e38 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.609691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e6945-f2df-4172-bdef-eec5cc386e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.609881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e6945-f2df-4172-bdef-eec5cc386e38 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.611169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e6945-f2df-4172-bdef-eec5cc386e38 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:06.611532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e6945-f2df-4172-bdef-eec5cc386e38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:06.612951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e6945-f2df-4172-bdef-eec5cc386e38 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:06.629294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:06.637305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 289/578] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Aug 1 11:51:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:06.637305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e6945-f2df-4172-bdef-eec5cc386e38 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.655001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620c536-d1a7-4e2d-b6fa-9fa1bb73cde6 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.689877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59300df-3a1e-422e-89c0-81857f8b0c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.690510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59300df-3a1e-422e-89c0-81857f8b0c7e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.690793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59300df-3a1e-422e-89c0-81857f8b0c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.691069 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59300df-3a1e-422e-89c0-81857f8b0c7e None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.691816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59300df-3a1e-422e-89c0-81857f8b0c7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:06.692123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59300df-3a1e-422e-89c0-81857f8b0c7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:06.693614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59300df-3a1e-422e-89c0-81857f8b0c7e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:06.701925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 290/579] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:06.702635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59300df-3a1e-422e-89c0-81857f8b0c7e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.745104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e6945-f2df-4172-bdef-eec5cc386e38 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.770042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:06.783178 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9d515-1e3d-44ea-88d5-0a01211f7156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.783838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9d515-1e3d-44ea-88d5-0a01211f7156 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.784121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9d515-1e3d-44ea-88d5-0a01211f7156 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.784425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf9d515-1e3d-44ea-88d5-0a01211f7156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.791628 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9d515-1e3d-44ea-88d5-0a01211f7156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:06.793118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf9d515-1e3d-44ea-88d5-0a01211f7156 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-ccf9d515-1e3d-44ea-88d5-0a01211f7156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:06.804594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:06.805508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:06.806399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:06.806399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:06.806399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:06.806399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:06.806399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:06.806399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:06.806399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:06.806399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:06.806399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 290/580] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:06.806880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9d515-1e3d-44ea-88d5-0a01211f7156 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.821215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59300df-3a1e-422e-89c0-81857f8b0c7e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.863512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90501d5b-d542-415a-ad48-d3a508c83f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.864242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90501d5b-d542-415a-ad48-d3a508c83f26 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.864552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90501d5b-d542-415a-ad48-d3a508c83f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.864819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90501d5b-d542-415a-ad48-d3a508c83f26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.879120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90501d5b-d542-415a-ad48-d3a508c83f26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-90501d5b-d542-415a-ad48-d3a508c83f26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:06.885002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:06.885922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:06.886814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 291/581] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:06.887493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90501d5b-d542-415a-ad48-d3a508c83f26 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:06.890047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:06.937578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf9d515-1e3d-44ea-88d5-0a01211f7156 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:06.972023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8f77e-efed-4d58-9e1c-914e61bbb324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:06.972736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8f77e-efed-4d58-9e1c-914e61bbb324 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:06.972984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8f77e-efed-4d58-9e1c-914e61bbb324 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:06.973256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8f77e-efed-4d58-9e1c-914e61bbb324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:06.984147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8f77e-efed-4d58-9e1c-914e61bbb324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:06.985982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8f77e-efed-4d58-9e1c-914e61bbb324 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:06.998195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90501d5b-d542-415a-ad48-d3a508c83f26 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-35e8f77e-efed-4d58-9e1c-914e61bbb324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:07.000182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:07.001158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:07.002060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:07.002060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:07.002060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:07.002060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:07.002060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 291/582] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:07.002349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8f77e-efed-4d58-9e1c-914e61bbb324 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.036168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa24b2-2734-4b29-b526-4e62a62d0c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:07.036860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa24b2-2734-4b29-b526-4e62a62d0c40 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:07.037145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa24b2-2734-4b29-b526-4e62a62d0c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:07.037428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa24b2-2734-4b29-b526-4e62a62d0c40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:07.047600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:07.048737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa24b2-2734-4b29-b526-4e62a62d0c40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:07.050233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa24b2-2734-4b29-b526-4e62a62d0c40 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-52aa24b2-2734-4b29-b526-4e62a62d0c40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:07.066334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:07.068157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:07.069204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:07.069204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:07.069204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:07.069204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:07.069204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 292/583] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:07.069204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa24b2-2734-4b29-b526-4e62a62d0c40 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.131366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8f77e-efed-4d58-9e1c-914e61bbb324 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:07.133966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:07.172038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bea00-c724-4e86-8621-db61149b2985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:07.173011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bea00-c724-4e86-8621-db61149b2985 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:07.173298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bea00-c724-4e86-8621-db61149b2985 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:07.173826 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665bea00-c724-4e86-8621-db61149b2985 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:07.174660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bea00-c724-4e86-8621-db61149b2985 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:07.175102 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bea00-c724-4e86-8621-db61149b2985 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:07.177165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665bea00-c724-4e86-8621-db61149b2985 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:07.181743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa24b2-2734-4b29-b526-4e62a62d0c40 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:07.189340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 292/584] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:07.191302 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bea00-c724-4e86-8621-db61149b2985 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.230877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7a829-12bb-413e-bb33-7a2f87afe192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:07.231510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7a829-12bb-413e-bb33-7a2f87afe192 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:07.231788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7a829-12bb-413e-bb33-7a2f87afe192 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:07.232058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7a829-12bb-413e-bb33-7a2f87afe192 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:07.232817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7a829-12bb-413e-bb33-7a2f87afe192 None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:07.233120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7a829-12bb-413e-bb33-7a2f87afe192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:07.234920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7a829-12bb-413e-bb33-7a2f87afe192 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:07.244931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 293/585] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:07.245613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7a829-12bb-413e-bb33-7a2f87afe192 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.302555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:07.310741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665bea00-c724-4e86-8621-db61149b2985 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:07.348049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fda127-4dcb-4d89-a7ed-4c5cedb34f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:07.348758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda127-4dcb-4d89-a7ed-4c5cedb34f3c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:07.349049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda127-4dcb-4d89-a7ed-4c5cedb34f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:07.349320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fda127-4dcb-4d89-a7ed-4c5cedb34f3c None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:07.350017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fda127-4dcb-4d89-a7ed-4c5cedb34f3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:07.350327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fda127-4dcb-4d89-a7ed-4c5cedb34f3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:07.351707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fda127-4dcb-4d89-a7ed-4c5cedb34f3c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:07.360075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7a829-12bb-413e-bb33-7a2f87afe192 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:07.369044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 293/586] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:07.369044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fda127-4dcb-4d89-a7ed-4c5cedb34f3c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.380920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:07.401473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ee0b2-d4af-4824-ad53-198bf2388c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:07.402116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ee0b2-d4af-4824-ad53-198bf2388c67 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:07.402407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ee0b2-d4af-4824-ad53-198bf2388c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:07.402680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ee0b2-d4af-4824-ad53-198bf2388c67 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:07.403384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ee0b2-d4af-4824-ad53-198bf2388c67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:07.403690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ee0b2-d4af-4824-ad53-198bf2388c67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:07.405094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ee0b2-d4af-4824-ad53-198bf2388c67 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:07.417697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 294/587] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:07.418512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ee0b2-d4af-4824-ad53-198bf2388c67 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.479397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fda127-4dcb-4d89-a7ed-4c5cedb34f3c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:07.516487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa7db6-79a5-4b07-9b42-2053264380e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:07.517190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa7db6-79a5-4b07-9b42-2053264380e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:07.517471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa7db6-79a5-4b07-9b42-2053264380e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:07.517746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa7db6-79a5-4b07-9b42-2053264380e5 None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/db3ff1b21c784e8cb5bffcb79f230d08/roles/a55004b069fa4cd99327dd3d4f55efcf` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:07.525280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ee0b2-d4af-4824-ad53-198bf2388c67 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:07.534236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa7db6-79a5-4b07-9b42-2053264380e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=db3ff1b21c784e8cb5bffcb79f230d08, role_id=a55004b069fa4cd99327dd3d4f55efcf)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:07.535736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa7db6-79a5-4b07-9b42-2053264380e5 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:07.560415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 294/588] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:51:07 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/db3ff1b21c784e8cb5bffcb79f230d08/roles/a55004b069fa4cd99327dd3d4f55efcf => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:51:07.560415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa7db6-79a5-4b07-9b42-2053264380e5 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.562645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:07.579803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1f781-a411-4db3-a661-b3eadd40b924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:07.580698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f781-a411-4db3-a661-b3eadd40b924 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:07.581080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f781-a411-4db3-a661-b3eadd40b924 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:07.581554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c1f781-a411-4db3-a661-b3eadd40b924 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:07.594332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1f781-a411-4db3-a661-b3eadd40b924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:07.596073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c1f781-a411-4db3-a661-b3eadd40b924 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-50c1f781-a411-4db3-a661-b3eadd40b924 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:07.614181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:07.615346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:07.616368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:07.616368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:07.616368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:07.616368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:07.622225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 295/589] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:07.623504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1f781-a411-4db3-a661-b3eadd40b924 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.660772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:07.695904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa7db6-79a5-4b07-9b42-2053264380e5 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:07.744490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc8e79-3048-4cc1-a8d6-ff161596cef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:07.745151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc8e79-3048-4cc1-a8d6-ff161596cef3 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:07.745429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc8e79-3048-4cc1-a8d6-ff161596cef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:07.745702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc8e79-3048-4cc1-a8d6-ff161596cef3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:07.753970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc8e79-3048-4cc1-a8d6-ff161596cef3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:07.755486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc8e79-3048-4cc1-a8d6-ff161596cef3 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:07.761220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c1f781-a411-4db3-a661-b3eadd40b924 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-9fbc8e79-3048-4cc1-a8d6-ff161596cef3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:07.770850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:07.772063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:07.773298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:07.773298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:07.773298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:07.773298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:07.773298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:07.773298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:07.773298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:07.773298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:07.773298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 295/590] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:07.773298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc8e79-3048-4cc1-a8d6-ff161596cef3 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.813526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edf648-24d4-4202-bd47-071ded92cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:07.814411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edf648-24d4-4202-bd47-071ded92cf3b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:07.814884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edf648-24d4-4202-bd47-071ded92cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:07.815287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edf648-24d4-4202-bd47-071ded92cf3b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:07.825608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edf648-24d4-4202-bd47-071ded92cf3b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:07.827788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edf648-24d4-4202-bd47-071ded92cf3b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-51edf648-24d4-4202-bd47-071ded92cf3b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:07.844099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:07.845343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:07.846494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:07.846494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:07.846494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:07.846494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:07.846494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:07.846494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:07.846494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:07.846494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:07.846494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 11:51:07.846494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:07.847819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:07.850145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 296/591] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:07.851071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edf648-24d4-4202-bd47-071ded92cf3b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.896049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc8e79-3048-4cc1-a8d6-ff161596cef3 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:07.933995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5cdfd-fac6-4daf-b97d-324b6e33391e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:07.934879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5cdfd-fac6-4daf-b97d-324b6e33391e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:07.934991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5cdfd-fac6-4daf-b97d-324b6e33391e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:07.935270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5cdfd-fac6-4daf-b97d-324b6e33391e None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:07.935985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5cdfd-fac6-4daf-b97d-324b6e33391e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:07.936303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5cdfd-fac6-4daf-b97d-324b6e33391e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:07.937707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5cdfd-fac6-4daf-b97d-324b6e33391e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:07.955606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 296/592] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:07.956655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5cdfd-fac6-4daf-b97d-324b6e33391e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:07.966904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:07.991991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edf648-24d4-4202-bd47-071ded92cf3b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:08.038090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f4e1-8bdf-46bd-b136-9ed24ce8c175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:08.038870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f4e1-8bdf-46bd-b136-9ed24ce8c175 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.039284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f4e1-8bdf-46bd-b136-9ed24ce8c175 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.039699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20f4e1-8bdf-46bd-b136-9ed24ce8c175 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.041124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20f4e1-8bdf-46bd-b136-9ed24ce8c175 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:08.041532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20f4e1-8bdf-46bd-b136-9ed24ce8c175 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:08.043560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f20f4e1-8bdf-46bd-b136-9ed24ce8c175 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:08.061731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 297/593] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:08.063032 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f4e1-8bdf-46bd-b136-9ed24ce8c175 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:08.073557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5cdfd-fac6-4daf-b97d-324b6e33391e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:08.105951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:08.122825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6268bd-cfff-4077-bee9-88710c68f3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:08.123634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6268bd-cfff-4077-bee9-88710c68f3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.124266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6268bd-cfff-4077-bee9-88710c68f3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.124669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6268bd-cfff-4077-bee9-88710c68f3c1 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.125800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6268bd-cfff-4077-bee9-88710c68f3c1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:08.126206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6268bd-cfff-4077-bee9-88710c68f3c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:08.128477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6268bd-cfff-4077-bee9-88710c68f3c1 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:08.156599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 297/594] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Aug 1 11:51:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:08.156599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6268bd-cfff-4077-bee9-88710c68f3c1 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:08.177793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20f4e1-8bdf-46bd-b136-9ed24ce8c175 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:08.214330 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86081a5d-353b-4534-ad9d-93f8963137f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:08.215378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081a5d-353b-4534-ad9d-93f8963137f4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.215862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081a5d-353b-4534-ad9d-93f8963137f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.216404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86081a5d-353b-4534-ad9d-93f8963137f4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.224862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86081a5d-353b-4534-ad9d-93f8963137f4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:08.226590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86081a5d-353b-4534-ad9d-93f8963137f4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-86081a5d-353b-4534-ad9d-93f8963137f4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:08.239741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:08.241001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:08.241924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:08.241924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:08.241924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:08.241924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:08.241924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:08.241924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:08.241924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:08.241924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:08.241924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 11:51:08.241924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:08.248979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 298/595] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:08.250182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86081a5d-353b-4534-ad9d-93f8963137f4 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:08.262690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:08.278584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6268bd-cfff-4077-bee9-88710c68f3c1 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:08.325886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514f292-a4c9-4729-a8ab-e8db93c3a33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:08.326730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514f292-a4c9-4729-a8ab-e8db93c3a33e None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.327012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514f292-a4c9-4729-a8ab-e8db93c3a33e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.327286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514f292-a4c9-4729-a8ab-e8db93c3a33e None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/184005a383ff4ab696d5235369b44628/roles/a55004b069fa4cd99327dd3d4f55efcf` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.346808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514f292-a4c9-4729-a8ab-e8db93c3a33e None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=184005a383ff4ab696d5235369b44628, role_id=a55004b069fa4cd99327dd3d4f55efcf)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:08.348937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514f292-a4c9-4729-a8ab-e8db93c3a33e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:08.364064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86081a5d-353b-4534-ad9d-93f8963137f4 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:08.366532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 298/596] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:51:08 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/184005a383ff4ab696d5235369b44628/roles/a55004b069fa4cd99327dd3d4f55efcf => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:51:08.369572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514f292-a4c9-4729-a8ab-e8db93c3a33e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:08.407419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af67ff4-e13e-4fca-a7ce-f82d27bca853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:08.408562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af67ff4-e13e-4fca-a7ce-f82d27bca853 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.409074 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af67ff4-e13e-4fca-a7ce-f82d27bca853 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.409579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af67ff4-e13e-4fca-a7ce-f82d27bca853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.412120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:08.425030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af67ff4-e13e-4fca-a7ce-f82d27bca853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:08.428254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af67ff4-e13e-4fca-a7ce-f82d27bca853 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-3af67ff4-e13e-4fca-a7ce-f82d27bca853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:08.446039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:08.447015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:08.447937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:08.447937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:08.447937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:08.447937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:08.450870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 299/597] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:08.457571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34468591-8669-4124-af38-d88de7a4efd8 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.458439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34468591-8669-4124-af38-d88de7a4efd8 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.459097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34468591-8669-4124-af38-d88de7a4efd8 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.461455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 300/598] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:08.462775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34468591-8669-4124-af38-d88de7a4efd8 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:08.508574 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514f292-a4c9-4729-a8ab-e8db93c3a33e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:08.541885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:08.560016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd2e5a-0742-4a4b-9d70-d96be0d196d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:08.560695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd2e5a-0742-4a4b-9d70-d96be0d196d7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.560970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd2e5a-0742-4a4b-9d70-d96be0d196d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.561238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd2e5a-0742-4a4b-9d70-d96be0d196d7 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.562045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd2e5a-0742-4a4b-9d70-d96be0d196d7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:08.562346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd2e5a-0742-4a4b-9d70-d96be0d196d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:08.563793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd2e5a-0742-4a4b-9d70-d96be0d196d7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:08.586356 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 299/599] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Aug 1 11:51:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:08.587342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04dd1a-35e2-4775-ada0-24ac9dcca7dc None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.587621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04dd1a-35e2-4775-ada0-24ac9dcca7dc None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.587889 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c04dd1a-35e2-4775-ada0-24ac9dcca7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.598682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34468591-8669-4124-af38-d88de7a4efd8 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:08.607578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-7c04dd1a-35e2-4775-ada0-24ac9dcca7dc None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:08.608333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7c04dd1a-35e2-4775-ada0-24ac9dcca7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:08.645446 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2fb53-b945-4b15-8f73-d9d5182e74fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:08.646320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2fb53-b945-4b15-8f73-d9d5182e74fa None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.646616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2fb53-b945-4b15-8f73-d9d5182e74fa None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.646938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2fb53-b945-4b15-8f73-d9d5182e74fa None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.647649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2fb53-b945-4b15-8f73-d9d5182e74fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:08.647948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2fb53-b945-4b15-8f73-d9d5182e74fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:08.650108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2fb53-b945-4b15-8f73-d9d5182e74fa None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:08.664152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 301/600] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:08.665725 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2fb53-b945-4b15-8f73-d9d5182e74fa None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:08.703935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 300/601] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:08.703935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04dd1a-35e2-4775-ada0-24ac9dcca7dc None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:08.764785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:08.806269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2fb53-b945-4b15-8f73-d9d5182e74fa None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:08.819761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c04dd1a-35e2-4775-ada0-24ac9dcca7dc None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:08.841063 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:08.861779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160904d-478b-4207-82b8-1dbf52898f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:08.862655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160904d-478b-4207-82b8-1dbf52898f7c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.863095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160904d-478b-4207-82b8-1dbf52898f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.863506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160904d-478b-4207-82b8-1dbf52898f7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.863714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef621-99e9-4d08-8a66-9a9dfc097150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:08.864533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef621-99e9-4d08-8a66-9a9dfc097150 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:08.864909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef621-99e9-4d08-8a66-9a9dfc097150 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:08.865279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef621-99e9-4d08-8a66-9a9dfc097150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:08.871494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160904d-478b-4207-82b8-1dbf52898f7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:08.873147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160904d-478b-4207-82b8-1dbf52898f7c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:08.877864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ef621-99e9-4d08-8a66-9a9dfc097150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:08.879780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ef621-99e9-4d08-8a66-9a9dfc097150 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-0160904d-478b-4207-82b8-1dbf52898f7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:08.885234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:08.886264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:08.887219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:08.887219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:08.887219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:08.887219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:08.887219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:08.887219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:08.887219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:08.887219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:08.889459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 301/602] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:08.890473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160904d-478b-4207-82b8-1dbf52898f7c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-403ef621-99e9-4d08-8a66-9a9dfc097150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:08.899173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:08.900083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:08.900984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:08.900984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:08.900984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:08.900984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:08.900984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 302/603] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:08.901678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef621-99e9-4d08-8a66-9a9dfc097150 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:08.984137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:09.009864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160904d-478b-4207-82b8-1dbf52898f7c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:09.035647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef621-99e9-4d08-8a66-9a9dfc097150 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:09.045727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edf74b-d7ea-4c25-b03e-ea51c206c5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:09.046789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edf74b-d7ea-4c25-b03e-ea51c206c5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.047256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edf74b-d7ea-4c25-b03e-ea51c206c5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.047804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edf74b-d7ea-4c25-b03e-ea51c206c5a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.059295 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1edf74b-d7ea-4c25-b03e-ea51c206c5a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-a1edf74b-d7ea-4c25-b03e-ea51c206c5a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:09.065399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:09.066325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:09.067197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:09.069931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 302/604] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:09.071005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edf74b-d7ea-4c25-b03e-ea51c206c5a4 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:09.074008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b47154-e760-496d-8326-120c8a040534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:09.074901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b47154-e760-496d-8326-120c8a040534 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.075208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b47154-e760-496d-8326-120c8a040534 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.076253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b47154-e760-496d-8326-120c8a040534 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.076382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b47154-e760-496d-8326-120c8a040534 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:09.076729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b47154-e760-496d-8326-120c8a040534 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:09.078289 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b47154-e760-496d-8326-120c8a040534 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:09.097333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 303/605] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:09.098197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b47154-e760-496d-8326-120c8a040534 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:09.132956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:09.189601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edf74b-d7ea-4c25-b03e-ea51c206c5a4 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:09.218542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b47154-e760-496d-8326-120c8a040534 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:09.241925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a518db-049c-4ab5-aac6-169de3e2c8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:09.242979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a518db-049c-4ab5-aac6-169de3e2c8cf None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.243452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a518db-049c-4ab5-aac6-169de3e2c8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.243915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a518db-049c-4ab5-aac6-169de3e2c8cf None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.244966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a518db-049c-4ab5-aac6-169de3e2c8cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:09.245573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a518db-049c-4ab5-aac6-169de3e2c8cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:09.247169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a518db-049c-4ab5-aac6-169de3e2c8cf None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:09.263350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0a07d-f0c9-4fd3-a138-f47a0a9d5be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:09.264045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a07d-f0c9-4fd3-a138-f47a0a9d5be8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.264602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a07d-f0c9-4fd3-a138-f47a0a9d5be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.264980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0a07d-f0c9-4fd3-a138-f47a0a9d5be8 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.266010 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0a07d-f0c9-4fd3-a138-f47a0a9d5be8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:09.266346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0a07d-f0c9-4fd3-a138-f47a0a9d5be8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:09.268123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0a07d-f0c9-4fd3-a138-f47a0a9d5be8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:09.272329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:09.285902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 303/606] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:09.285902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a518db-049c-4ab5-aac6-169de3e2c8cf None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:09.289110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 304/607] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:09.290880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0a07d-f0c9-4fd3-a138-f47a0a9d5be8 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:09.405752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:09.414402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0a07d-f0c9-4fd3-a138-f47a0a9d5be8 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:09.452363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0c3ee-4592-43af-95aa-c1ccec96beca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:09.453807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0c3ee-4592-43af-95aa-c1ccec96beca None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.454325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0c3ee-4592-43af-95aa-c1ccec96beca None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.454795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0c3ee-4592-43af-95aa-c1ccec96beca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.468060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0c3ee-4592-43af-95aa-c1ccec96beca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:09.469865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0c3ee-4592-43af-95aa-c1ccec96beca None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:09.470487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a518db-049c-4ab5-aac6-169de3e2c8cf None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-a0f0c3ee-4592-43af-95aa-c1ccec96beca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:09.486445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:09.487422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:09.488314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:09.488314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:09.488314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:09.488314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:09.490673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 305/608] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:09.491726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0c3ee-4592-43af-95aa-c1ccec96beca None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:09.534136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f58d53-f095-47de-aa4f-9c046cb829d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:09.534831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f58d53-f095-47de-aa4f-9c046cb829d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.535107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f58d53-f095-47de-aa4f-9c046cb829d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.535383 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f58d53-f095-47de-aa4f-9c046cb829d1 None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/dc0102a256934641a999dd091e6ba0b3/roles/a55004b069fa4cd99327dd3d4f55efcf` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.580055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f58d53-f095-47de-aa4f-9c046cb829d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=dc0102a256934641a999dd091e6ba0b3, role_id=a55004b069fa4cd99327dd3d4f55efcf)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:09.581452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f58d53-f095-47de-aa4f-9c046cb829d1 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:09.608929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0c3ee-4592-43af-95aa-c1ccec96beca None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:09.624317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 304/609] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:51:09 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/dc0102a256934641a999dd091e6ba0b3/roles/a55004b069fa4cd99327dd3d4f55efcf => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:51:09.624317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f58d53-f095-47de-aa4f-9c046cb829d1 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:09.682842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a0b58-4e70-4615-9872-c331b85e3682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:09.683424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a0b58-4e70-4615-9872-c331b85e3682 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.683700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a0b58-4e70-4615-9872-c331b85e3682 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.683969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a0b58-4e70-4615-9872-c331b85e3682 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.691940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a0b58-4e70-4615-9872-c331b85e3682 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:09.696110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a0b58-4e70-4615-9872-c331b85e3682 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-567a0b58-4e70-4615-9872-c331b85e3682 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:09.717167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:09.718136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:09.718997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:09.718997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:09.718997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:09.718997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:09.718997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:09.718997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:09.718997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:09.718997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:09.719890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 306/610] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:09.721700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dfab9-4c94-4c37-ae64-0f64a8d75bc7 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.723543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dfab9-4c94-4c37-ae64-0f64a8d75bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.723543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dfab9-4c94-4c37-ae64-0f64a8d75bc7 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.726087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 307/611] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:09.726815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41dfab9-4c94-4c37-ae64-0f64a8d75bc7 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:09.727347 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:09.755464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:09.837074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f58d53-f095-47de-aa4f-9c046cb829d1 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:09.892241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386e83c-7931-4a68-aeb1-d962ab223242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:09.893406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386e83c-7931-4a68-aeb1-d962ab223242 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.893698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386e83c-7931-4a68-aeb1-d962ab223242 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.896331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386e83c-7931-4a68-aeb1-d962ab223242 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.896790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386e83c-7931-4a68-aeb1-d962ab223242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:09.897254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386e83c-7931-4a68-aeb1-d962ab223242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:09.899456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386e83c-7931-4a68-aeb1-d962ab223242 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:09.905225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41dfab9-4c94-4c37-ae64-0f64a8d75bc7 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:09.911852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 305/612] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:09.913339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221d5d5-35b3-47cf-b322-8ef7d84b52e1 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.913627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221d5d5-35b3-47cf-b322-8ef7d84b52e1 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.913905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221d5d5-35b3-47cf-b322-8ef7d84b52e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.935133 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-2221d5d5-35b3-47cf-b322-8ef7d84b52e1 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:09.936273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2221d5d5-35b3-47cf-b322-8ef7d84b52e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:09.954470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8829dd-f37a-4951-b948-938b8304a32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:09.955416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8829dd-f37a-4951-b948-938b8304a32a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:09.956085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8829dd-f37a-4951-b948-938b8304a32a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:09.956388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8829dd-f37a-4951-b948-938b8304a32a None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:09.957160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8829dd-f37a-4951-b948-938b8304a32a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:09.957458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8829dd-f37a-4951-b948-938b8304a32a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:09.959719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8829dd-f37a-4951-b948-938b8304a32a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:09.977256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 308/613] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:09.978121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8829dd-f37a-4951-b948-938b8304a32a None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:10.021440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 306/614] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:10.022409 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221d5d5-35b3-47cf-b322-8ef7d84b52e1 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:10.070686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:10.084193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:10.109463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8829dd-f37a-4951-b948-938b8304a32a None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:10.145477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53972797-acc1-4e2e-b647-ca38f236b229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:10.146240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53972797-acc1-4e2e-b647-ca38f236b229 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:10.146592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53972797-acc1-4e2e-b647-ca38f236b229 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:10.146957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53972797-acc1-4e2e-b647-ca38f236b229 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:10.151511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221d5d5-35b3-47cf-b322-8ef7d84b52e1 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:10.154790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53972797-acc1-4e2e-b647-ca38f236b229 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:10.156450 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53972797-acc1-4e2e-b647-ca38f236b229 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-53972797-acc1-4e2e-b647-ca38f236b229 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:10.169168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:10.170358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:10.171525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:10.171525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:10.171525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:10.171525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:10.171525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:10.171525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:10.171525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:10.171525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:10.171525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:10.171525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:10.172873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 309/615] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:10.173713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53972797-acc1-4e2e-b647-ca38f236b229 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:10.195252 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bc62d-caf5-4080-8a2e-97e2901942fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:10.195885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bc62d-caf5-4080-8a2e-97e2901942fa None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:10.196170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bc62d-caf5-4080-8a2e-97e2901942fa None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:10.196499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bc62d-caf5-4080-8a2e-97e2901942fa None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/8dfbda2838b242558c885f913fafb5d8/roles/a55004b069fa4cd99327dd3d4f55efcf` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:10.212236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bc62d-caf5-4080-8a2e-97e2901942fa None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=8dfbda2838b242558c885f913fafb5d8, role_id=a55004b069fa4cd99327dd3d4f55efcf)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:10.213813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bc62d-caf5-4080-8a2e-97e2901942fa None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:10.231300 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 307/616] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:51:10 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/8dfbda2838b242558c885f913fafb5d8/roles/a55004b069fa4cd99327dd3d4f55efcf => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:51:10.232207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bc62d-caf5-4080-8a2e-97e2901942fa None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:10.333862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53972797-acc1-4e2e-b647-ca38f236b229 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:10.347045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bc62d-caf5-4080-8a2e-97e2901942fa None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:10.360868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:10.394425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:10.417750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159967d8-a309-4b9d-9a1d-cb3045d09858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:10.417969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec566e9e-389f-4a87-a6f8-b2048b678b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:10.418428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159967d8-a309-4b9d-9a1d-cb3045d09858 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:10.418708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159967d8-a309-4b9d-9a1d-cb3045d09858 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:10.418886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec566e9e-389f-4a87-a6f8-b2048b678b16 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:10.418978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159967d8-a309-4b9d-9a1d-cb3045d09858 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:10.419149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec566e9e-389f-4a87-a6f8-b2048b678b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:10.419428 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec566e9e-389f-4a87-a6f8-b2048b678b16 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:10.420376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec566e9e-389f-4a87-a6f8-b2048b678b16 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:10.420783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec566e9e-389f-4a87-a6f8-b2048b678b16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:10.422345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec566e9e-389f-4a87-a6f8-b2048b678b16 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:10.443633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 308/617] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Aug 1 11:51:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:10.449396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec566e9e-389f-4a87-a6f8-b2048b678b16 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:10.453182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159967d8-a309-4b9d-9a1d-cb3045d09858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-159967d8-a309-4b9d-9a1d-cb3045d09858 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:10.466861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:10.467842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:10.468708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:10.477120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 310/618] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:10.490017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159967d8-a309-4b9d-9a1d-cb3045d09858 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:10.611955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec566e9e-389f-4a87-a6f8-b2048b678b16 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:10.634943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159967d8-a309-4b9d-9a1d-cb3045d09858 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:10.649370 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8ec3a-2e69-4c9c-9b6e-d706b24b2692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:10.650024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ec3a-2e69-4c9c-9b6e-d706b24b2692 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:10.650301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ec3a-2e69-4c9c-9b6e-d706b24b2692 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:10.650687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8ec3a-2e69-4c9c-9b6e-d706b24b2692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:10.661607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8ec3a-2e69-4c9c-9b6e-d706b24b2692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:10.663276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8ec3a-2e69-4c9c-9b6e-d706b24b2692 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:10.672377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:10.675713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cc292-584c-4a48-8fc3-45a13fcb9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:10.676641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cc292-584c-4a48-8fc3-45a13fcb9d3d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:10.677123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cc292-584c-4a48-8fc3-45a13fcb9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:10.677538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cc292-584c-4a48-8fc3-45a13fcb9d3d None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:10.678405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cc292-584c-4a48-8fc3-45a13fcb9d3d None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:10.678854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cc292-584c-4a48-8fc3-45a13fcb9d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:10.680499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cc292-584c-4a48-8fc3-45a13fcb9d3d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-8da8ec3a-2e69-4c9c-9b6e-d706b24b2692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:10.680840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:10.682044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:10.682941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:10.682941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:10.682941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:10.682941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:10.682941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 309/619] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:10.683589 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61e833-39ae-412f-bcf9-80d42d777305 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:10.683870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61e833-39ae-412f-bcf9-80d42d777305 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:10.684137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61e833-39ae-412f-bcf9-80d42d777305 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:10.685715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 310/620] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:10.690514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61e833-39ae-412f-bcf9-80d42d777305 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:10.696013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 311/621] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:10.702160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b325a47-12ca-4bd0-9c37-82d7ee761689 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:10.702160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b325a47-12ca-4bd0-9c37-82d7ee761689 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:10.702160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b325a47-12ca-4bd0-9c37-82d7ee761689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:10.731549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2b325a47-12ca-4bd0-9c37-82d7ee761689 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:10.733120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2b325a47-12ca-4bd0-9c37-82d7ee761689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:10.795108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:10.819038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61e833-39ae-412f-bcf9-80d42d777305 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:10.821376 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 312/622] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:10.822515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b325a47-12ca-4bd0-9c37-82d7ee761689 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:10.881858 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf1dff-b37e-4a3f-a44d-6d484e5424e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:10.882776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf1dff-b37e-4a3f-a44d-6d484e5424e2 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:10.883214 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf1dff-b37e-4a3f-a44d-6d484e5424e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:10.883549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf1dff-b37e-4a3f-a44d-6d484e5424e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:10.884333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf1dff-b37e-4a3f-a44d-6d484e5424e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:10.884638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf1dff-b37e-4a3f-a44d-6d484e5424e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:10.886053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf1dff-b37e-4a3f-a44d-6d484e5424e2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:10.899247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 311/623] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:10.900165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf1dff-b37e-4a3f-a44d-6d484e5424e2 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:10.964752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:10.966000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b325a47-12ca-4bd0-9c37-82d7ee761689 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.010074 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb55194-d6d8-4093-8853-056c29528a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.011289 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb55194-d6d8-4093-8853-056c29528a9e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.011758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb55194-d6d8-4093-8853-056c29528a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.012238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb55194-d6d8-4093-8853-056c29528a9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.012506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf1dff-b37e-4a3f-a44d-6d484e5424e2 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.025858 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb55194-d6d8-4093-8853-056c29528a9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:11.027334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb55194-d6d8-4093-8853-056c29528a9e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-5eb55194-d6d8-4093-8853-056c29528a9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:11.049891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:11.050852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:11.051731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:11.051731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:11.051731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:11.051731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:11.056227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 313/624] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:11.057120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb55194-d6d8-4093-8853-056c29528a9e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:11.063036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3225cc4-27b2-4eb0-be99-7ba453db7d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.063991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3225cc4-27b2-4eb0-be99-7ba453db7d80 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.064508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3225cc4-27b2-4eb0-be99-7ba453db7d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.065072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3225cc4-27b2-4eb0-be99-7ba453db7d80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.080216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3225cc4-27b2-4eb0-be99-7ba453db7d80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-d3225cc4-27b2-4eb0-be99-7ba453db7d80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:11.087716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:11.088739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:11.089594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:11.090235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 312/625] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:11.096425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3225cc4-27b2-4eb0-be99-7ba453db7d80 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:11.099151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:11.189315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb55194-d6d8-4093-8853-056c29528a9e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.216752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3225cc4-27b2-4eb0-be99-7ba453db7d80 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.227383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2ff33-05b2-46e1-a32c-5a5facc5664e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.228492 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2ff33-05b2-46e1-a32c-5a5facc5664e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.229103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2ff33-05b2-46e1-a32c-5a5facc5664e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.229581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2ff33-05b2-46e1-a32c-5a5facc5664e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.238113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:11.238861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2ff33-05b2-46e1-a32c-5a5facc5664e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:11.240845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2ff33-05b2-46e1-a32c-5a5facc5664e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:11.253472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3010c8-81c7-4c66-9605-fcb1e040b62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.254099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3010c8-81c7-4c66-9605-fcb1e040b62d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.254441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3010c8-81c7-4c66-9605-fcb1e040b62d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.254735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3010c8-81c7-4c66-9605-fcb1e040b62d None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.255435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3010c8-81c7-4c66-9605-fcb1e040b62d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:11.255736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3010c8-81c7-4c66-9605-fcb1e040b62d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-35e2ff33-05b2-46e1-a32c-5a5facc5664e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:11.256285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:11.257195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:11.258093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:11.258093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:11.258093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:11.258093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:11.258093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:11.258093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:11.258093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:11.258093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:11.258527 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3010c8-81c7-4c66-9605-fcb1e040b62d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:11.260421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 314/626] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:11.261470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2ff33-05b2-46e1-a32c-5a5facc5664e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:11.274234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 313/627] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:11.275248 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3010c8-81c7-4c66-9605-fcb1e040b62d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:11.363165 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:11.375650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2ff33-05b2-46e1-a32c-5a5facc5664e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.376520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3010c8-81c7-4c66-9605-fcb1e040b62d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.415912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca9edd-7266-4248-bc3a-bae1439336ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.417606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca9edd-7266-4248-bc3a-bae1439336ff None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.418198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca9edd-7266-4248-bc3a-bae1439336ff None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.418749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca9edd-7266-4248-bc3a-bae1439336ff None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.419892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca9edd-7266-4248-bc3a-bae1439336ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:11.420533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca9edd-7266-4248-bc3a-bae1439336ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:11.421662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8350cd4-6e23-40ee-8de0-4478ce82bb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.422430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8350cd4-6e23-40ee-8de0-4478ce82bb9e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.422720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca9edd-7266-4248-bc3a-bae1439336ff None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:11.422875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8350cd4-6e23-40ee-8de0-4478ce82bb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.423239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8350cd4-6e23-40ee-8de0-4478ce82bb9e None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.424050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8350cd4-6e23-40ee-8de0-4478ce82bb9e None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:11.424490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8350cd4-6e23-40ee-8de0-4478ce82bb9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:11.426149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8350cd4-6e23-40ee-8de0-4478ce82bb9e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:11.441539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 315/628] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:11.441539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8350cd4-6e23-40ee-8de0-4478ce82bb9e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:11.442648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 314/629] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:11.443636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca9edd-7266-4248-bc3a-bae1439336ff None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:11.512777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:11.548689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca9edd-7266-4248-bc3a-bae1439336ff None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.554941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8350cd4-6e23-40ee-8de0-4478ce82bb9e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.584471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e54dfc-d1fb-4053-a97d-f0bec25a1999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.585131 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e54dfc-d1fb-4053-a97d-f0bec25a1999 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.585412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e54dfc-d1fb-4053-a97d-f0bec25a1999 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.585687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e54dfc-d1fb-4053-a97d-f0bec25a1999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.594640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8ce42-a2e7-4bd8-b05a-2695fb5a4ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.595599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8ce42-a2e7-4bd8-b05a-2695fb5a4ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.596095 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8ce42-a2e7-4bd8-b05a-2695fb5a4ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.596597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8ce42-a2e7-4bd8-b05a-2695fb5a4ba9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.596705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e54dfc-d1fb-4053-a97d-f0bec25a1999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:11.598134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e54dfc-d1fb-4053-a97d-f0bec25a1999 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:11.606193 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8ce42-a2e7-4bd8-b05a-2695fb5a4ba9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:11.608049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8ce42-a2e7-4bd8-b05a-2695fb5a4ba9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-85e54dfc-d1fb-4053-a97d-f0bec25a1999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:11.614281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:11.615157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:11.616069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:11.616069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:11.616069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:11.616069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:11.620188 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 315/630] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:11.621042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e54dfc-d1fb-4053-a97d-f0bec25a1999 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-c4b8ce42-a2e7-4bd8-b05a-2695fb5a4ba9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:11.623935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:11.624888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:11.625813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:11.625813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:11.625813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:11.625813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:11.625813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:11.625813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:11.625813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:11.625813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:11.625813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:11.625813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:11.628249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 316/631] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Aug 1 11:51:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:11.629951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8ce42-a2e7-4bd8-b05a-2695fb5a4ba9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:11.647708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:11.720405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:11.729764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e54dfc-d1fb-4053-a97d-f0bec25a1999 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.746469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8ce42-a2e7-4bd8-b05a-2695fb5a4ba9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.765567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae43fd0-f65a-4cd2-bbc9-f5e79ff3a110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.766222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae43fd0-f65a-4cd2-bbc9-f5e79ff3a110 None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.766507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae43fd0-f65a-4cd2-bbc9-f5e79ff3a110 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.766773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae43fd0-f65a-4cd2-bbc9-f5e79ff3a110 None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/b9b6a96421e349a88a38590cfce167fb/roles/a55004b069fa4cd99327dd3d4f55efcf` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.785459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae43fd0-f65a-4cd2-bbc9-f5e79ff3a110 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=b9b6a96421e349a88a38590cfce167fb, role_id=a55004b069fa4cd99327dd3d4f55efcf)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:11.787091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae43fd0-f65a-4cd2-bbc9-f5e79ff3a110 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:11.787846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d72766-ca18-44a5-aa92-895ea42e5d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.788720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d72766-ca18-44a5-aa92-895ea42e5d71 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.789294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d72766-ca18-44a5-aa92-895ea42e5d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.789775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d72766-ca18-44a5-aa92-895ea42e5d71 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.790741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d72766-ca18-44a5-aa92-895ea42e5d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:11.791421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d72766-ca18-44a5-aa92-895ea42e5d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:11.793161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d72766-ca18-44a5-aa92-895ea42e5d71 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:11.803520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 316/632] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:51:11 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/b9b6a96421e349a88a38590cfce167fb/roles/a55004b069fa4cd99327dd3d4f55efcf => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:51:11.804926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae43fd0-f65a-4cd2-bbc9-f5e79ff3a110 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:11.812001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 317/633] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:11.817770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d72766-ca18-44a5-aa92-895ea42e5d71 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:11.932901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:11.938796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae43fd0-f65a-4cd2-bbc9-f5e79ff3a110 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.978751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d72766-ca18-44a5-aa92-895ea42e5d71 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:11.981186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c4874-182e-4dba-b41a-c7ce47b7e47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:11.981948 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c4874-182e-4dba-b41a-c7ce47b7e47e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:11.982297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c4874-182e-4dba-b41a-c7ce47b7e47e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:11.982639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c4874-182e-4dba-b41a-c7ce47b7e47e None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:11.983553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c4874-182e-4dba-b41a-c7ce47b7e47e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:11.986367 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c4874-182e-4dba-b41a-c7ce47b7e47e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:11.986367 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c4874-182e-4dba-b41a-c7ce47b7e47e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:12.003291 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:12.007226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 317/634] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Aug 1 11:51:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:12.008780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f3583-f510-4183-98e5-d9a0b916c24f None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.010409 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f3583-f510-4183-98e5-d9a0b916c24f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.011117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f3583-f510-4183-98e5-d9a0b916c24f None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.012855 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 318/635] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:12.013899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f3583-f510-4183-98e5-d9a0b916c24f None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:12.044395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62ceb9-949d-43ad-9632-d956b6df1f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:12.045098 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62ceb9-949d-43ad-9632-d956b6df1f84 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.045404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62ceb9-949d-43ad-9632-d956b6df1f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.045782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62ceb9-949d-43ad-9632-d956b6df1f84 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.055697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62ceb9-949d-43ad-9632-d956b6df1f84 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:12.058195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa62ceb9-949d-43ad-9632-d956b6df1f84 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-fa62ceb9-949d-43ad-9632-d956b6df1f84 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:12.074276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:12.075263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:12.076207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 318/636] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:12.077377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6179c-8f36-4a49-b2d1-cd126ef9f028 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.077377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6179c-8f36-4a49-b2d1-cd126ef9f028 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.077540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6179c-8f36-4a49-b2d1-cd126ef9f028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.106731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-f3c6179c-8f36-4a49-b2d1-cd126ef9f028 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:12.108353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f3c6179c-8f36-4a49-b2d1-cd126ef9f028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:12.149282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f3583-f510-4183-98e5-d9a0b916c24f None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:12.180116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 319/637] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:12.181535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6179c-8f36-4a49-b2d1-cd126ef9f028 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:12.201696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1714ed7-22bd-4903-89c5-fb4c301ae47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:12.202362 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1714ed7-22bd-4903-89c5-fb4c301ae47d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.202641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1714ed7-22bd-4903-89c5-fb4c301ae47d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.202911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1714ed7-22bd-4903-89c5-fb4c301ae47d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.261244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:12.291949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:12.294538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1714ed7-22bd-4903-89c5-fb4c301ae47d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:12.296283 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1714ed7-22bd-4903-89c5-fb4c301ae47d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:12.311495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6179c-8f36-4a49-b2d1-cd126ef9f028 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:12.357554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37bc1b-07d6-46f3-a00d-3985b7c0d2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:12.358741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37bc1b-07d6-46f3-a00d-3985b7c0d2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.359581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37bc1b-07d6-46f3-a00d-3985b7c0d2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.360719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37bc1b-07d6-46f3-a00d-3985b7c0d2c9 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.361753 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37bc1b-07d6-46f3-a00d-3985b7c0d2c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:12.362455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37bc1b-07d6-46f3-a00d-3985b7c0d2c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:12.364836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37bc1b-07d6-46f3-a00d-3985b7c0d2c9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:12.393123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 320/638] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Aug 1 11:51:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:12.393123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37bc1b-07d6-46f3-a00d-3985b7c0d2c9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:12.420596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 319/639] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 01 11:51:12.421805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1714ed7-22bd-4903-89c5-fb4c301ae47d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:12.527337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37bc1b-07d6-46f3-a00d-3985b7c0d2c9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:12.535275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:12.538435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1714ed7-22bd-4903-89c5-fb4c301ae47d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:12.553499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:12.568539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af7cca-f267-4f68-87df-9c4373e06010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:12.569222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af7cca-f267-4f68-87df-9c4373e06010 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.569467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af7cca-f267-4f68-87df-9c4373e06010 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.569740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af7cca-f267-4f68-87df-9c4373e06010 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.576323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b5d56-b4bf-452c-9194-02148dcb3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:12.576952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b5d56-b4bf-452c-9194-02148dcb3b92 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.577227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b5d56-b4bf-452c-9194-02148dcb3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.577497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b5d56-b4bf-452c-9194-02148dcb3b92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.580517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af7cca-f267-4f68-87df-9c4373e06010 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-04af7cca-f267-4f68-87df-9c4373e06010 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:12.585659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:12.586715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:12.587528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:12.592436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 321/640] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:12.593871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af7cca-f267-4f68-87df-9c4373e06010 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:12.655983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b5d56-b4bf-452c-9194-02148dcb3b92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:12.657806 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b5d56-b4bf-452c-9194-02148dcb3b92 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:12.700150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af7cca-f267-4f68-87df-9c4373e06010 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:12.737098 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a33e-974a-4d2e-96e8-2e65cb239214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:12.737743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a33e-974a-4d2e-96e8-2e65cb239214 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.738017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a33e-974a-4d2e-96e8-2e65cb239214 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.738282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2a33e-974a-4d2e-96e8-2e65cb239214 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.749545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2a33e-974a-4d2e-96e8-2e65cb239214 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:12.751006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2a33e-974a-4d2e-96e8-2e65cb239214 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:12.752842 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:12.761641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 320/641] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:51:12.763010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b04bcf-4275-469e-ac34-b0fe02e0c653 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.763299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b04bcf-4275-469e-ac34-b0fe02e0c653 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.763573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b04bcf-4275-469e-ac34-b0fe02e0c653 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.769034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 321/642] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:12.769034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b04bcf-4275-469e-ac34-b0fe02e0c653 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-36f2a33e-974a-4d2e-96e8-2e65cb239214 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:12.769239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:12.770220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:12.771104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:12.771104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:12.771104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:12.771104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:12.771104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 322/643] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:12.771104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a33e-974a-4d2e-96e8-2e65cb239214 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:12.806516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:12.886047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2a33e-974a-4d2e-96e8-2e65cb239214 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:12.887567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b04bcf-4275-469e-ac34-b0fe02e0c653 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:12.922154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5f83b-8eea-435a-baf3-ec3f109b6aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:12.922801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5f83b-8eea-435a-baf3-ec3f109b6aaf None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.923079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5f83b-8eea-435a-baf3-ec3f109b6aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.923178 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf7a0a-2f5e-4015-9325-bc1c065d9521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:12.923377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5f83b-8eea-435a-baf3-ec3f109b6aaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.923777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf7a0a-2f5e-4015-9325-bc1c065d9521 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.924111 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf7a0a-2f5e-4015-9325-bc1c065d9521 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.924414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf7a0a-2f5e-4015-9325-bc1c065d9521 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.925184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf7a0a-2f5e-4015-9325-bc1c065d9521 None admin] RBAC: Adding query filter params (name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:12.925484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf7a0a-2f5e-4015-9325-bc1c065d9521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:12.926939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf7a0a-2f5e-4015-9325-bc1c065d9521 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:12.934617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5f83b-8eea-435a-baf3-ec3f109b6aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:12.935972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 322/644] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:12.936079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5f83b-8eea-435a-baf3-ec3f109b6aaf None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:12.937116 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccc536-986f-42b6-9cd4-6fb765fd3d73 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:12.937406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccc536-986f-42b6-9cd4-6fb765fd3d73 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:12.937680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccc536-986f-42b6-9cd4-6fb765fd3d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-8ed5f83b-8eea-435a-baf3-ec3f109b6aaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:12.955396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:12.956680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:12.957874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:12.957874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:12.957874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:12.957874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:12.958105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-88ccc536-986f-42b6-9cd4-6fb765fd3d73 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:12.958105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-88ccc536-986f-42b6-9cd4-6fb765fd3d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:12.961282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 323/645] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:12.962060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5f83b-8eea-435a-baf3-ec3f109b6aaf None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.022298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:13.035171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 323/646] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:13.036228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ccc536-986f-42b6-9cd4-6fb765fd3d73 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.064561 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:13.081746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5f83b-8eea-435a-baf3-ec3f109b6aaf None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.117933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8423f-6317-49c9-adea-b889dae9a3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:13.118559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8423f-6317-49c9-adea-b889dae9a3fd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:13.118830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8423f-6317-49c9-adea-b889dae9a3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:13.119120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8423f-6317-49c9-adea-b889dae9a3fd None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:13.119810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8423f-6317-49c9-adea-b889dae9a3fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:13.120115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8423f-6317-49c9-adea-b889dae9a3fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:13.121582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8423f-6317-49c9-adea-b889dae9a3fd None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:13.139603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 324/647] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:13.139603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8423f-6317-49c9-adea-b889dae9a3fd None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.150099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ccc536-986f-42b6-9cd4-6fb765fd3d73 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.187968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91e07f-5398-4ee4-a602-f2d24d252b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:13.188654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91e07f-5398-4ee4-a602-f2d24d252b5e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:13.188978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91e07f-5398-4ee4-a602-f2d24d252b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:13.189199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91e07f-5398-4ee4-a602-f2d24d252b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:13.199355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91e07f-5398-4ee4-a602-f2d24d252b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:13.200851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91e07f-5398-4ee4-a602-f2d24d252b5e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-8e91e07f-5398-4ee4-a602-f2d24d252b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:13.220052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:13.221045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:13.221973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:13.221973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:13.221973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:13.221973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:13.222685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 324/648] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:13.223185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91e07f-5398-4ee4-a602-f2d24d252b5e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.246541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8423f-6317-49c9-adea-b889dae9a3fd None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.272593 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:13.288922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70ae1c-cdce-46c7-a4fd-9d722bcafd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:13.289866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70ae1c-cdce-46c7-a4fd-9d722bcafd09 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:13.290279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70ae1c-cdce-46c7-a4fd-9d722bcafd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:13.290731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70ae1c-cdce-46c7-a4fd-9d722bcafd09 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:13.291447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70ae1c-cdce-46c7-a4fd-9d722bcafd09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:13.291755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70ae1c-cdce-46c7-a4fd-9d722bcafd09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:13.293335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70ae1c-cdce-46c7-a4fd-9d722bcafd09 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:13.312041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 325/649] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:13.312041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70ae1c-cdce-46c7-a4fd-9d722bcafd09 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.317531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:13.329313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91e07f-5398-4ee4-a602-f2d24d252b5e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.370619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99835e00-aa8a-4cb6-abf0-2a95dfd446d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:13.371277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99835e00-aa8a-4cb6-abf0-2a95dfd446d9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:13.371558 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99835e00-aa8a-4cb6-abf0-2a95dfd446d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:13.372065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99835e00-aa8a-4cb6-abf0-2a95dfd446d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:13.382369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99835e00-aa8a-4cb6-abf0-2a95dfd446d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-99835e00-aa8a-4cb6-abf0-2a95dfd446d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:13.387723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:13.388915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:13.389944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:13.393905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 325/650] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:13.394778 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99835e00-aa8a-4cb6-abf0-2a95dfd446d9 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.430489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70ae1c-cdce-46c7-a4fd-9d722bcafd09 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.469918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450c7be-d02c-4287-bab2-a26d42f74a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:13.470582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450c7be-d02c-4287-bab2-a26d42f74a84 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:13.470859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450c7be-d02c-4287-bab2-a26d42f74a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:13.471128 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450c7be-d02c-4287-bab2-a26d42f74a84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:13.478649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450c7be-d02c-4287-bab2-a26d42f74a84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:13.480167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450c7be-d02c-4287-bab2-a26d42f74a84 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-5450c7be-d02c-4287-bab2-a26d42f74a84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:13.492504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:13.493620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:13.499829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:13.499829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:13.499829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:13.499829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:13.499829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:13.499829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:13.499829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:13.499829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:13.499829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 326/651] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:13.499829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450c7be-d02c-4287-bab2-a26d42f74a84 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.514924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99835e00-aa8a-4cb6-abf0-2a95dfd446d9 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.518242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:13.554565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596282a-20aa-40b6-a0dd-bd6f343bf3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:13.555374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596282a-20aa-40b6-a0dd-bd6f343bf3dd None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:13.555656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596282a-20aa-40b6-a0dd-bd6f343bf3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:13.555932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596282a-20aa-40b6-a0dd-bd6f343bf3dd None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:13.556666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596282a-20aa-40b6-a0dd-bd6f343bf3dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:13.556969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596282a-20aa-40b6-a0dd-bd6f343bf3dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:13.558462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596282a-20aa-40b6-a0dd-bd6f343bf3dd None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:13.565119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:13.576415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 326/652] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:13.576415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596282a-20aa-40b6-a0dd-bd6f343bf3dd None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.612394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450c7be-d02c-4287-bab2-a26d42f74a84 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.648069 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f4a75-22bf-420c-a87d-1cea75f52009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:13.648903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f4a75-22bf-420c-a87d-1cea75f52009 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:13.649303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f4a75-22bf-420c-a87d-1cea75f52009 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:13.649677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f4a75-22bf-420c-a87d-1cea75f52009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:13.656638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f4a75-22bf-420c-a87d-1cea75f52009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:13.658178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f4a75-22bf-420c-a87d-1cea75f52009 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-af9f4a75-22bf-420c-a87d-1cea75f52009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:13.669737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:13.670655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:13.671625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:13.671625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:13.671625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:13.671625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:13.671625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:13.671625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:13.671625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:13.671625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:13.671625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 327/653] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:13.671625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f4a75-22bf-420c-a87d-1cea75f52009 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.683609 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596282a-20aa-40b6-a0dd-bd6f343bf3dd None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.720113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8254-68cb-4092-a7c3-c7e4325c365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:13.720920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8254-68cb-4092-a7c3-c7e4325c365e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:13.721204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8254-68cb-4092-a7c3-c7e4325c365e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:13.721482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c8254-68cb-4092-a7c3-c7e4325c365e None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:13.722207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8254-68cb-4092-a7c3-c7e4325c365e None admin] RBAC: Adding query filter params (name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:13.722514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8254-68cb-4092-a7c3-c7e4325c365e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:13.723945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73c8254-68cb-4092-a7c3-c7e4325c365e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:13.737241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 327/654] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:13.737241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8254-68cb-4092-a7c3-c7e4325c365e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.766276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:13.789109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f4a75-22bf-420c-a87d-1cea75f52009 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.813371 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:13.826365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f754f-5771-481a-8ba1-57d78a31d5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:13.827198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f754f-5771-481a-8ba1-57d78a31d5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:13.830860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f754f-5771-481a-8ba1-57d78a31d5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:13.830860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f754f-5771-481a-8ba1-57d78a31d5a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:13.830860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f754f-5771-481a-8ba1-57d78a31d5a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:13.830860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f754f-5771-481a-8ba1-57d78a31d5a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:13.831597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f754f-5771-481a-8ba1-57d78a31d5a8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:13.843784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c8254-68cb-4092-a7c3-c7e4325c365e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.854553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 328/655] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:13.854553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f754f-5771-481a-8ba1-57d78a31d5a8 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.881778 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff698cf9-8fe3-4fb4-866a-193fed43a7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:13.882514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff698cf9-8fe3-4fb4-866a-193fed43a7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:13.882791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff698cf9-8fe3-4fb4-866a-193fed43a7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:13.883061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff698cf9-8fe3-4fb4-866a-193fed43a7d0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:13.890782 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff698cf9-8fe3-4fb4-866a-193fed43a7d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:13.892213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff698cf9-8fe3-4fb4-866a-193fed43a7d0 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-ff698cf9-8fe3-4fb4-866a-193fed43a7d0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:13.903992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:13.904984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 328/656] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:13.905912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff698cf9-8fe3-4fb4-866a-193fed43a7d0 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:13.959976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f754f-5771-481a-8ba1-57d78a31d5a8 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:13.994166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:13.999897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e32e97-9434-46c0-88a9-5f7861e2909e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:14.000628 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e32e97-9434-46c0-88a9-5f7861e2909e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.000912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e32e97-9434-46c0-88a9-5f7861e2909e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.001195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e32e97-9434-46c0-88a9-5f7861e2909e None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.001972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e32e97-9434-46c0-88a9-5f7861e2909e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:14.002277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e32e97-9434-46c0-88a9-5f7861e2909e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:14.003738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e32e97-9434-46c0-88a9-5f7861e2909e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:14.009360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff698cf9-8fe3-4fb4-866a-193fed43a7d0 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:14.015826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 329/657] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:14.016666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e32e97-9434-46c0-88a9-5f7861e2909e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:14.049495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621be789-32cd-4cd0-af57-17812f559b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:14.050149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621be789-32cd-4cd0-af57-17812f559b18 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.050429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621be789-32cd-4cd0-af57-17812f559b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.050703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621be789-32cd-4cd0-af57-17812f559b18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.061450 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:14.062054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621be789-32cd-4cd0-af57-17812f559b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:14.063638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621be789-32cd-4cd0-af57-17812f559b18 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-621be789-32cd-4cd0-af57-17812f559b18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:14.078247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:14.079266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:14.080125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:14.080125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:14.080125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:14.080125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:14.080125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 329/658] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:14.080731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326761be-3447-4023-b215-f0596804d9b5 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.081015 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326761be-3447-4023-b215-f0596804d9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.081287 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326761be-3447-4023-b215-f0596804d9b5 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.082594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 330/659] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:14.084386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326761be-3447-4023-b215-f0596804d9b5 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:14.123192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e32e97-9434-46c0-88a9-5f7861e2909e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:14.157556 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e73f0b-4398-4a23-8759-db42dfc08cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:14.158174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e73f0b-4398-4a23-8759-db42dfc08cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.158451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e73f0b-4398-4a23-8759-db42dfc08cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.158721 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e73f0b-4398-4a23-8759-db42dfc08cf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.159506 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e73f0b-4398-4a23-8759-db42dfc08cf5 None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=db3ff1b21c784e8cb5bffcb79f230d08) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:14.159819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e73f0b-4398-4a23-8759-db42dfc08cf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:14.161374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e73f0b-4398-4a23-8759-db42dfc08cf5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:14.167743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 330/660] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:14 2022] GET /identity/v3/role_assignments?user.id=db3ff1b21c784e8cb5bffcb79f230d08&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:14.168594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e73f0b-4398-4a23-8759-db42dfc08cf5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:14.197353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326761be-3447-4023-b215-f0596804d9b5 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:14.239017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee45e30-20fb-4e3d-8868-169f82826817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:14.239673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee45e30-20fb-4e3d-8868-169f82826817 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.239954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee45e30-20fb-4e3d-8868-169f82826817 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.240378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee45e30-20fb-4e3d-8868-169f82826817 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.241212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee45e30-20fb-4e3d-8868-169f82826817 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:14.241521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee45e30-20fb-4e3d-8868-169f82826817 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:14.243012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee45e30-20fb-4e3d-8868-169f82826817 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:14.253067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:14.264138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 331/661] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Aug 1 11:51:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:14.265055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee45e30-20fb-4e3d-8868-169f82826817 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:14.314543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e73f0b-4398-4a23-8759-db42dfc08cf5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:14.325638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:14.354774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d1753-8b13-48b2-a149-d90b55b6933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:14.355409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d1753-8b13-48b2-a149-d90b55b6933e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.355687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d1753-8b13-48b2-a149-d90b55b6933e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.356257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d1753-8b13-48b2-a149-d90b55b6933e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.356845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d1753-8b13-48b2-a149-d90b55b6933e None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=184005a383ff4ab696d5235369b44628) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:14.357664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d1753-8b13-48b2-a149-d90b55b6933e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:14.358603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d1753-8b13-48b2-a149-d90b55b6933e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:14.366615 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 331/662] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:14 2022] GET /identity/v3/role_assignments?user.id=184005a383ff4ab696d5235369b44628&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:14.368209 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f853c47-2762-484b-9413-e843b15f8224 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.368566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f853c47-2762-484b-9413-e843b15f8224 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.368991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f853c47-2762-484b-9413-e843b15f8224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.389488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7f853c47-2762-484b-9413-e843b15f8224 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:14.391111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7f853c47-2762-484b-9413-e843b15f8224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:14.394310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee45e30-20fb-4e3d-8868-169f82826817 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:14.443725 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55ebc0-6e2c-42dc-b2a7-e28dd6e20d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:14.444413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55ebc0-6e2c-42dc-b2a7-e28dd6e20d4f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.444697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55ebc0-6e2c-42dc-b2a7-e28dd6e20d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.445174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55ebc0-6e2c-42dc-b2a7-e28dd6e20d4f None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.445877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55ebc0-6e2c-42dc-b2a7-e28dd6e20d4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:14.446958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55ebc0-6e2c-42dc-b2a7-e28dd6e20d4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:14.447615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55ebc0-6e2c-42dc-b2a7-e28dd6e20d4f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:14.460692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 332/663] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:14.461577 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55ebc0-6e2c-42dc-b2a7-e28dd6e20d4f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:14.489338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 332/664] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:14.490256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f853c47-2762-484b-9413-e843b15f8224 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:14.551544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:14.610705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55ebc0-6e2c-42dc-b2a7-e28dd6e20d4f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:14.636378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f853c47-2762-484b-9413-e843b15f8224 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:14.638427 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:14.660105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9b98f-5ad7-4080-bcb1-d86ac3652e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:14.660927 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9b98f-5ad7-4080-bcb1-d86ac3652e1f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.661208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9b98f-5ad7-4080-bcb1-d86ac3652e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.661481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9b98f-5ad7-4080-bcb1-d86ac3652e1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.682259 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86a8c5-4266-4ba0-bd23-019cdf74aac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:14.683060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86a8c5-4266-4ba0-bd23-019cdf74aac2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.683523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86a8c5-4266-4ba0-bd23-019cdf74aac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.683798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86a8c5-4266-4ba0-bd23-019cdf74aac2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.693938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86a8c5-4266-4ba0-bd23-019cdf74aac2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:14.695619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86a8c5-4266-4ba0-bd23-019cdf74aac2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-0b86a8c5-4266-4ba0-bd23-019cdf74aac2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:14.710675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:14.711789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 333/665] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:14.713064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86a8c5-4266-4ba0-bd23-019cdf74aac2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:14.760153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9b98f-5ad7-4080-bcb1-d86ac3652e1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:14.762543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9b98f-5ad7-4080-bcb1-d86ac3652e1f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:14.825716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:14.838706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86a8c5-4266-4ba0-bd23-019cdf74aac2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:14.890981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 333/666] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:51:14.891793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf86e5-91ef-4f3b-a818-26e9896f2634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:14.892320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9b98f-5ad7-4080-bcb1-d86ac3652e1f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:14.892681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf86e5-91ef-4f3b-a818-26e9896f2634 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:14.893071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf86e5-91ef-4f3b-a818-26e9896f2634 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:14.893411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf86e5-91ef-4f3b-a818-26e9896f2634 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:14.907789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf86e5-91ef-4f3b-a818-26e9896f2634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-06cf86e5-91ef-4f3b-a818-26e9896f2634 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:14.915170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:14.916151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 334/667] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:14.921971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf86e5-91ef-4f3b-a818-26e9896f2634 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:14.949009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:15.011946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9b98f-5ad7-4080-bcb1-d86ac3652e1f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:15.054948 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67de7d-91c4-4006-b872-c80507ac7a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:15.055415 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf86e5-91ef-4f3b-a818-26e9896f2634 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:15.055752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67de7d-91c4-4006-b872-c80507ac7a5a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:15.056104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67de7d-91c4-4006-b872-c80507ac7a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:15.056491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67de7d-91c4-4006-b872-c80507ac7a5a None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:15.057372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67de7d-91c4-4006-b872-c80507ac7a5a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:15.057760 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67de7d-91c4-4006-b872-c80507ac7a5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:15.059282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e67de7d-91c4-4006-b872-c80507ac7a5a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:15.082554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 334/668] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Aug 1 11:51:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:15.083603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67de7d-91c4-4006-b872-c80507ac7a5a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:15.098474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8170d25-96ae-4d98-9124-6d8746f667f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:15.099150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8170d25-96ae-4d98-9124-6d8746f667f6 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:15.099649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8170d25-96ae-4d98-9124-6d8746f667f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:15.099925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8170d25-96ae-4d98-9124-6d8746f667f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:15.100900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:15.117869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8170d25-96ae-4d98-9124-6d8746f667f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:15.120121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8170d25-96ae-4d98-9124-6d8746f667f6 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-b8170d25-96ae-4d98-9124-6d8746f667f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:15.137604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:15.138778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:15.139917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:15.139917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:15.139917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:15.139917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:15.141108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 335/669] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:15.141669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8170d25-96ae-4d98-9124-6d8746f667f6 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:15.221900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67de7d-91c4-4006-b872-c80507ac7a5a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:15.227068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:15.261349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea748e-4281-4e07-b916-97c060cec91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:15.262082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea748e-4281-4e07-b916-97c060cec91d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:15.262297 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8170d25-96ae-4d98-9124-6d8746f667f6 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:15.262418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea748e-4281-4e07-b916-97c060cec91d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:15.262759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea748e-4281-4e07-b916-97c060cec91d None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:15.263550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea748e-4281-4e07-b916-97c060cec91d None admin] RBAC: Adding query filter params (name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:15.264028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea748e-4281-4e07-b916-97c060cec91d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:15.266267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea748e-4281-4e07-b916-97c060cec91d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:15.278250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 335/670] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:15.279247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea748e-4281-4e07-b916-97c060cec91d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:15.306942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c574-89e9-4c77-bf90-e67d9d8536d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:15.308528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c574-89e9-4c77-bf90-e67d9d8536d1 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:15.308992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c574-89e9-4c77-bf90-e67d9d8536d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:15.309677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d1c574-89e9-4c77-bf90-e67d9d8536d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:15.387978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c574-89e9-4c77-bf90-e67d9d8536d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:15.389864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d1c574-89e9-4c77-bf90-e67d9d8536d1 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:15.476124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:15.504861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:15.508630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 336/671] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:51:15.509976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c574-89e9-4c77-bf90-e67d9d8536d1 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:15.536318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea748e-4281-4e07-b916-97c060cec91d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:15.574634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79735bd3-a994-425c-8b08-1e89c1ece1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:15.575482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79735bd3-a994-425c-8b08-1e89c1ece1ae None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:15.576107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79735bd3-a994-425c-8b08-1e89c1ece1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:15.576232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79735bd3-a994-425c-8b08-1e89c1ece1ae None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:15.576910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79735bd3-a994-425c-8b08-1e89c1ece1ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:15.577274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79735bd3-a994-425c-8b08-1e89c1ece1ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:15.578871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79735bd3-a994-425c-8b08-1e89c1ece1ae None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:15.592977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 336/672] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:15.594035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79735bd3-a994-425c-8b08-1e89c1ece1ae None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:15.645782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d1c574-89e9-4c77-bf90-e67d9d8536d1 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:15.682623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a9b65-50ec-4c24-bbc7-5ccdaf5ba6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:15.683311 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a9b65-50ec-4c24-bbc7-5ccdaf5ba6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:15.683588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a9b65-50ec-4c24-bbc7-5ccdaf5ba6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:15.683856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a9b65-50ec-4c24-bbc7-5ccdaf5ba6e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:15.696052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a9b65-50ec-4c24-bbc7-5ccdaf5ba6e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:15.697571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a9b65-50ec-4c24-bbc7-5ccdaf5ba6e2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-510a9b65-50ec-4c24-bbc7-5ccdaf5ba6e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:15.714254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:15.715200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:15.716167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:15.716167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:15.716167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:15.716167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:15.716167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 337/673] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:15.716912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a9b65-50ec-4c24-bbc7-5ccdaf5ba6e2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:15.723030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79735bd3-a994-425c-8b08-1e89c1ece1ae None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:15.747180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:15.767485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8385543-888f-4006-9059-0c4647e6dbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:15.768573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8385543-888f-4006-9059-0c4647e6dbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:15.768999 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8385543-888f-4006-9059-0c4647e6dbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:15.769430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8385543-888f-4006-9059-0c4647e6dbf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:15.773479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:15.785221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8385543-888f-4006-9059-0c4647e6dbf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:15.787467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8385543-888f-4006-9059-0c4647e6dbf4 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-f8385543-888f-4006-9059-0c4647e6dbf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:15.806759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:15.807901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:15.809066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:15.809066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:15.809066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:15.809066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:15.815981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 337/674] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:15.815981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8385543-888f-4006-9059-0c4647e6dbf4 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:15.841951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a9b65-50ec-4c24-bbc7-5ccdaf5ba6e2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:15.887444 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293673-4a4d-47cb-9189-f4d832197292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:15.888797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293673-4a4d-47cb-9189-f4d832197292 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:15.889499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293673-4a4d-47cb-9189-f4d832197292 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:15.889983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33293673-4a4d-47cb-9189-f4d832197292 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:15.899736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33293673-4a4d-47cb-9189-f4d832197292 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:15.902382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33293673-4a4d-47cb-9189-f4d832197292 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-33293673-4a4d-47cb-9189-f4d832197292 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:15.917963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:15.918929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:15.919810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:15.919810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:15.919810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:15.919810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:15.919810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:15.919810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:15.919810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:15.919810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:15.926812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 338/675] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:15.926812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293673-4a4d-47cb-9189-f4d832197292 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:15.942322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8385543-888f-4006-9059-0c4647e6dbf4 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:15.987383 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae335b-eb1b-475e-9ea8-e9e3f1c70e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:15.988966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae335b-eb1b-475e-9ea8-e9e3f1c70e7f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:15.989741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae335b-eb1b-475e-9ea8-e9e3f1c70e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:15.990219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae335b-eb1b-475e-9ea8-e9e3f1c70e7f None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:15.991291 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae335b-eb1b-475e-9ea8-e9e3f1c70e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:15.992085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae335b-eb1b-475e-9ea8-e9e3f1c70e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:15.995241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae335b-eb1b-475e-9ea8-e9e3f1c70e7f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:16.009459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 338/676] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:16.010259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae335b-eb1b-475e-9ea8-e9e3f1c70e7f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:16.024888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:16.046044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33293673-4a4d-47cb-9189-f4d832197292 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:16.065624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:16.094217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267165e2-4005-40ba-a6c8-df3b3c14934b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:16.095259 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267165e2-4005-40ba-a6c8-df3b3c14934b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.095753 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267165e2-4005-40ba-a6c8-df3b3c14934b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.096244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267165e2-4005-40ba-a6c8-df3b3c14934b None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.097817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267165e2-4005-40ba-a6c8-df3b3c14934b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:16.098437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267165e2-4005-40ba-a6c8-df3b3c14934b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:16.100933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267165e2-4005-40ba-a6c8-df3b3c14934b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:16.121473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 339/677] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:16.121473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267165e2-4005-40ba-a6c8-df3b3c14934b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:16.131468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae335b-eb1b-475e-9ea8-e9e3f1c70e7f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:16.172984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a934247-61ed-4a7a-b7d3-2ae1b0fc60fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:16.174388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a934247-61ed-4a7a-b7d3-2ae1b0fc60fa None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.174877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a934247-61ed-4a7a-b7d3-2ae1b0fc60fa None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.175391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a934247-61ed-4a7a-b7d3-2ae1b0fc60fa None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.176757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a934247-61ed-4a7a-b7d3-2ae1b0fc60fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:16.177603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a934247-61ed-4a7a-b7d3-2ae1b0fc60fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:16.179626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a934247-61ed-4a7a-b7d3-2ae1b0fc60fa None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:16.201961 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 339/678] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:16.201961 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a934247-61ed-4a7a-b7d3-2ae1b0fc60fa None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:16.248803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267165e2-4005-40ba-a6c8-df3b3c14934b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:16.285830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860167d-f525-4499-981c-dfb4652e2e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:16.286405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860167d-f525-4499-981c-dfb4652e2e4e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.286682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860167d-f525-4499-981c-dfb4652e2e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.286954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860167d-f525-4499-981c-dfb4652e2e4e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.295003 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860167d-f525-4499-981c-dfb4652e2e4e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:16.296516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860167d-f525-4499-981c-dfb4652e2e4e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-0860167d-f525-4499-981c-dfb4652e2e4e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:16.309115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:16.310106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:16.311044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 340/679] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:16.311669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860167d-f525-4499-981c-dfb4652e2e4e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:16.325257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:16.329371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a934247-61ed-4a7a-b7d3-2ae1b0fc60fa None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:16.340052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:16.368732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff6d87-6c46-47c7-9f8d-51a4f42a9cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:16.369723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff6d87-6c46-47c7-9f8d-51a4f42a9cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.370186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff6d87-6c46-47c7-9f8d-51a4f42a9cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.370654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff6d87-6c46-47c7-9f8d-51a4f42a9cd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.377963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff6d87-6c46-47c7-9f8d-51a4f42a9cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:16.379589 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff6d87-6c46-47c7-9f8d-51a4f42a9cd0 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-7aff6d87-6c46-47c7-9f8d-51a4f42a9cd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:16.391704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:16.392631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:16.393513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:16.393513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:16.393513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:16.393513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:16.393513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:16.393513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:16.393513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:16.393513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:16.396634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 340/680] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:16.397673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff6d87-6c46-47c7-9f8d-51a4f42a9cd0 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:16.427193 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860167d-f525-4499-981c-dfb4652e2e4e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:16.468814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96990e5-f257-4d4a-bc99-703eda25b870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:16.469437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96990e5-f257-4d4a-bc99-703eda25b870 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.470065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96990e5-f257-4d4a-bc99-703eda25b870 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.470065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96990e5-f257-4d4a-bc99-703eda25b870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.470769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96990e5-f257-4d4a-bc99-703eda25b870 None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=dc0102a256934641a999dd091e6ba0b3) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:16.471162 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96990e5-f257-4d4a-bc99-703eda25b870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:16.472542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96990e5-f257-4d4a-bc99-703eda25b870 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:16.478893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 341/681] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:16 2022] GET /identity/v3/role_assignments?user.id=dc0102a256934641a999dd091e6ba0b3&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:16.479966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11762b-6eaa-4ec7-9ee4-9f7779bdad93 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.480255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11762b-6eaa-4ec7-9ee4-9f7779bdad93 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.480560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11762b-6eaa-4ec7-9ee4-9f7779bdad93 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.481946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 342/682] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:16.485819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11762b-6eaa-4ec7-9ee4-9f7779bdad93 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:16.517277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff6d87-6c46-47c7-9f8d-51a4f42a9cd0 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:16.560258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47050e4f-da3f-404d-8767-5cbcea642842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:16.560815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47050e4f-da3f-404d-8767-5cbcea642842 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.561094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47050e4f-da3f-404d-8767-5cbcea642842 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.561370 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47050e4f-da3f-404d-8767-5cbcea642842 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.562568 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47050e4f-da3f-404d-8767-5cbcea642842 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:16.563172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47050e4f-da3f-404d-8767-5cbcea642842 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:16.564984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47050e4f-da3f-404d-8767-5cbcea642842 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:16.587014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 341/683] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Aug 1 11:51:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:16.588403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ac295-4945-4699-93a1-2e6071006684 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.588690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ac295-4945-4699-93a1-2e6071006684 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.588963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ac295-4945-4699-93a1-2e6071006684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.605534 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:16.610485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-5b4ac295-4945-4699-93a1-2e6071006684 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:16.611399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5b4ac295-4945-4699-93a1-2e6071006684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:16.636557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:16.654320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11762b-6eaa-4ec7-9ee4-9f7779bdad93 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:16.690499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 342/684] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:16.691132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a61fcb-e0bc-4067-884a-a77241b94d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:16.691806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61fcb-e0bc-4067-884a-a77241b94d8c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.692087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61fcb-e0bc-4067-884a-a77241b94d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.692238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ac295-4945-4699-93a1-2e6071006684 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:16.692496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a61fcb-e0bc-4067-884a-a77241b94d8c None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.693186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a61fcb-e0bc-4067-884a-a77241b94d8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:16.693493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a61fcb-e0bc-4067-884a-a77241b94d8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:16.694951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a61fcb-e0bc-4067-884a-a77241b94d8c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:16.708686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 343/685] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:16.709977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95675068-38d1-4ab2-8b6e-1ad74b77f1e5 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.710243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95675068-38d1-4ab2-8b6e-1ad74b77f1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.710515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95675068-38d1-4ab2-8b6e-1ad74b77f1e5 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.711859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 344/686] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:16.716806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95675068-38d1-4ab2-8b6e-1ad74b77f1e5 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:16.803003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ac295-4945-4699-93a1-2e6071006684 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:16.836589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95675068-38d1-4ab2-8b6e-1ad74b77f1e5 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:16.844489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3915a-d33c-4b56-9299-da2560d7c356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:16.845460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3915a-d33c-4b56-9299-da2560d7c356 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.845967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3915a-d33c-4b56-9299-da2560d7c356 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.846443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3915a-d33c-4b56-9299-da2560d7c356 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.870055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:16.875390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:16.880230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc19932-0624-4a16-9211-2ff8c6381bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:16.881166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc19932-0624-4a16-9211-2ff8c6381bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.881455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc19932-0624-4a16-9211-2ff8c6381bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.881733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc19932-0624-4a16-9211-2ff8c6381bc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.883192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc19932-0624-4a16-9211-2ff8c6381bc2 None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=8dfbda2838b242558c885f913fafb5d8) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:16.883512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc19932-0624-4a16-9211-2ff8c6381bc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:16.885252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc19932-0624-4a16-9211-2ff8c6381bc2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:16.894689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 345/687] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:16 2022] GET /identity/v3/role_assignments?user.id=8dfbda2838b242558c885f913fafb5d8&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:16.896215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d36201-7f47-4738-b2d7-91dde529f636 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:16.896624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d36201-7f47-4738-b2d7-91dde529f636 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:16.896960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d36201-7f47-4738-b2d7-91dde529f636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:16.918578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-23d36201-7f47-4738-b2d7-91dde529f636 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:16.919944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-23d36201-7f47-4738-b2d7-91dde529f636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:16.947005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3915a-d33c-4b56-9299-da2560d7c356 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:16.949809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3915a-d33c-4b56-9299-da2560d7c356 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:17.007608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 346/688] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:17.008784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d36201-7f47-4738-b2d7-91dde529f636 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:17.093246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 343/689] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:51:17.094636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3915a-d33c-4b56-9299-da2560d7c356 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:17.142605 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d36201-7f47-4738-b2d7-91dde529f636 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:17.162446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:17.171602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:17.184845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb005c76-a9dc-4fde-843e-77d5987347e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:17.185589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb005c76-a9dc-4fde-843e-77d5987347e2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:17.185933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb005c76-a9dc-4fde-843e-77d5987347e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:17.186269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb005c76-a9dc-4fde-843e-77d5987347e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:17.199423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb005c76-a9dc-4fde-843e-77d5987347e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-fb005c76-a9dc-4fde-843e-77d5987347e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:17.206308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:17.207296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:17.208143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:17.208827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 347/690] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:17.211284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb005c76-a9dc-4fde-843e-77d5987347e2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:17.222012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3915a-d33c-4b56-9299-da2560d7c356 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:17.256883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c3c84-48a0-4d93-a93d-fb3d51942788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:17.257666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c3c84-48a0-4d93-a93d-fb3d51942788 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:17.257949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c3c84-48a0-4d93-a93d-fb3d51942788 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:17.258221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c3c84-48a0-4d93-a93d-fb3d51942788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:17.268707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c3c84-48a0-4d93-a93d-fb3d51942788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:17.270266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c3c84-48a0-4d93-a93d-fb3d51942788 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-402c3c84-48a0-4d93-a93d-fb3d51942788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:17.286230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:17.287176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:17.288050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:17.288050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:17.288050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:17.288050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:17.288050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 344/691] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:17.288546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c3c84-48a0-4d93-a93d-fb3d51942788 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:17.322714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb005c76-a9dc-4fde-843e-77d5987347e2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:17.361885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524fab3-74ac-4d38-b0ac-e00663028192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:17.362620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524fab3-74ac-4d38-b0ac-e00663028192 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:17.362908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524fab3-74ac-4d38-b0ac-e00663028192 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:17.363315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524fab3-74ac-4d38-b0ac-e00663028192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:17.374485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524fab3-74ac-4d38-b0ac-e00663028192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:17.376191 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524fab3-74ac-4d38-b0ac-e00663028192 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-1524fab3-74ac-4d38-b0ac-e00663028192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:17.392352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:17.394377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:17.396626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:17.396626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:17.396626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:17.396626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:17.396626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 348/692] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:17.397279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c3c84-48a0-4d93-a93d-fb3d51942788 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:17.397433 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524fab3-74ac-4d38-b0ac-e00663028192 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:17.403944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:17.431000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686549d8-8bd8-40b9-ad30-245c76a82c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:17.431632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686549d8-8bd8-40b9-ad30-245c76a82c19 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:17.431904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686549d8-8bd8-40b9-ad30-245c76a82c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:17.432169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686549d8-8bd8-40b9-ad30-245c76a82c19 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:17.432925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686549d8-8bd8-40b9-ad30-245c76a82c19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:17.433225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686549d8-8bd8-40b9-ad30-245c76a82c19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:17.434701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686549d8-8bd8-40b9-ad30-245c76a82c19 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:17.435459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:17.450539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 345/693] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:17.451815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686549d8-8bd8-40b9-ad30-245c76a82c19 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:17.517895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524fab3-74ac-4d38-b0ac-e00663028192 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:17.562321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efb2ae-8297-4664-9be0-f891d60d6364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:17.563354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efb2ae-8297-4664-9be0-f891d60d6364 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:17.563730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efb2ae-8297-4664-9be0-f891d60d6364 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:17.564529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efb2ae-8297-4664-9be0-f891d60d6364 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:17.565621 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efb2ae-8297-4664-9be0-f891d60d6364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:17.566027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efb2ae-8297-4664-9be0-f891d60d6364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:17.568186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efb2ae-8297-4664-9be0-f891d60d6364 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:17.570932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686549d8-8bd8-40b9-ad30-245c76a82c19 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:17.583247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 349/694] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:17.588326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efb2ae-8297-4664-9be0-f891d60d6364 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:17.619385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84cf11-532c-409f-b58f-cd8d291a544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:17.620406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84cf11-532c-409f-b58f-cd8d291a544e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:17.620921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84cf11-532c-409f-b58f-cd8d291a544e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:17.621390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84cf11-532c-409f-b58f-cd8d291a544e None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:17.622294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84cf11-532c-409f-b58f-cd8d291a544e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:17.622811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84cf11-532c-409f-b58f-cd8d291a544e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:17.624422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84cf11-532c-409f-b58f-cd8d291a544e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:17.637664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 346/695] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:17.638829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84cf11-532c-409f-b58f-cd8d291a544e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:17.670758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:17.696418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:17.705315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efb2ae-8297-4664-9be0-f891d60d6364 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:17.743395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8876d-8f17-4e87-9ce0-edf33257a0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:17.744338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8876d-8f17-4e87-9ce0-edf33257a0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:17.744616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8876d-8f17-4e87-9ce0-edf33257a0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:17.744885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8876d-8f17-4e87-9ce0-edf33257a0e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:17.749733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84cf11-532c-409f-b58f-cd8d291a544e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:17.756429 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8876d-8f17-4e87-9ce0-edf33257a0e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:17.757894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f8876d-8f17-4e87-9ce0-edf33257a0e4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-a1f8876d-8f17-4e87-9ce0-edf33257a0e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:17.774083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:17.775254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:17.776111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:17.776111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:17.776111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:17.776111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:17.780319 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 350/696] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:17.781250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8876d-8f17-4e87-9ce0-edf33257a0e4 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:17.791433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80657b-a995-4673-9ec4-0ca9db304388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:17.792424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80657b-a995-4673-9ec4-0ca9db304388 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:17.792940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80657b-a995-4673-9ec4-0ca9db304388 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:17.793403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80657b-a995-4673-9ec4-0ca9db304388 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:17.794287 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80657b-a995-4673-9ec4-0ca9db304388 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:17.795924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80657b-a995-4673-9ec4-0ca9db304388 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:17.824436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 347/697] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:51:17 2022] POST /identity/v3/users => generated 254 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:17.825737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80657b-a995-4673-9ec4-0ca9db304388 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:17.904660 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f8876d-8f17-4e87-9ce0-edf33257a0e4 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:17.935710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:17.949158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8544568-23bf-4f96-9fe3-b0fd30b912f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:17.949158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8544568-23bf-4f96-9fe3-b0fd30b912f4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:17.949158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8544568-23bf-4f96-9fe3-b0fd30b912f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:17.949158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8544568-23bf-4f96-9fe3-b0fd30b912f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:17.955251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8544568-23bf-4f96-9fe3-b0fd30b912f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:17.957143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8544568-23bf-4f96-9fe3-b0fd30b912f4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:17.970110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80657b-a995-4673-9ec4-0ca9db304388 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-b8544568-23bf-4f96-9fe3-b0fd30b912f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:17.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:17.971485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:17.972377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:17.972377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:17.972377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:17.972377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:17.972377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:17.972377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:17.972377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:17.972377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:17.972377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 351/698] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:17.981996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8544568-23bf-4f96-9fe3-b0fd30b912f4 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:18.004020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:18.012598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335515b-10b8-4f2a-9799-4a7fb641b979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:18.013519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335515b-10b8-4f2a-9799-4a7fb641b979 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:18.013958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335515b-10b8-4f2a-9799-4a7fb641b979 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:18.014416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335515b-10b8-4f2a-9799-4a7fb641b979 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:18.015250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335515b-10b8-4f2a-9799-4a7fb641b979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:18.015681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335515b-10b8-4f2a-9799-4a7fb641b979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:18.017661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335515b-10b8-4f2a-9799-4a7fb641b979 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:18.032520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 348/699] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:18.038069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335515b-10b8-4f2a-9799-4a7fb641b979 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:18.121846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8544568-23bf-4f96-9fe3-b0fd30b912f4 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:18.147391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335515b-10b8-4f2a-9799-4a7fb641b979 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:18.161106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5572dfb-b363-45d3-887f-3b8ca78a1d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:18.161666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5572dfb-b363-45d3-887f-3b8ca78a1d10 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:18.161940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5572dfb-b363-45d3-887f-3b8ca78a1d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:18.162212 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5572dfb-b363-45d3-887f-3b8ca78a1d10 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:18.162974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5572dfb-b363-45d3-887f-3b8ca78a1d10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:18.163278 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5572dfb-b363-45d3-887f-3b8ca78a1d10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:18.165251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5572dfb-b363-45d3-887f-3b8ca78a1d10 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:18.185340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 352/700] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:18.186368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5572dfb-b363-45d3-887f-3b8ca78a1d10 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:18.188792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea7277-c8d5-464d-8a7c-3f6a953a0b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:18.189754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea7277-c8d5-464d-8a7c-3f6a953a0b70 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:18.190282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea7277-c8d5-464d-8a7c-3f6a953a0b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:18.190704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea7277-c8d5-464d-8a7c-3f6a953a0b70 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:18.199452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea7277-c8d5-464d-8a7c-3f6a953a0b70 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:18.201228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea7277-c8d5-464d-8a7c-3f6a953a0b70 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-fcea7277-c8d5-464d-8a7c-3f6a953a0b70 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:18.214618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:18.215799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:18.216862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:18.216862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:18.216862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:18.216862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:18.216862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:18.216862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:18.216862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:18.216862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:18.216862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:18.216862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:18.219249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 349/701] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Aug 1 11:51:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:18.224932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea7277-c8d5-464d-8a7c-3f6a953a0b70 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:18.236896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:18.262526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:18.301758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5572dfb-b363-45d3-887f-3b8ca78a1d10 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:18.341467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4d3ff-b89d-43e7-bb51-a5552d01b4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:18.342426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4d3ff-b89d-43e7-bb51-a5552d01b4ac None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:18.342883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4d3ff-b89d-43e7-bb51-a5552d01b4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:18.343322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4d3ff-b89d-43e7-bb51-a5552d01b4ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:18.344314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4d3ff-b89d-43e7-bb51-a5552d01b4ac None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=b9b6a96421e349a88a38590cfce167fb) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:18.344816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4d3ff-b89d-43e7-bb51-a5552d01b4ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:18.346431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4d3ff-b89d-43e7-bb51-a5552d01b4ac None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:18.352274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea7277-c8d5-464d-8a7c-3f6a953a0b70 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:18.353503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 353/702] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:18 2022] GET /identity/v3/role_assignments?user.id=b9b6a96421e349a88a38590cfce167fb&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:18.392392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce14c38-133d-4526-9a9c-a14b9fd12bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:18.393044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce14c38-133d-4526-9a9c-a14b9fd12bff None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:18.393322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce14c38-133d-4526-9a9c-a14b9fd12bff None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:18.393595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce14c38-133d-4526-9a9c-a14b9fd12bff None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:18.394415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce14c38-133d-4526-9a9c-a14b9fd12bff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:18.394771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce14c38-133d-4526-9a9c-a14b9fd12bff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:18.396357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce14c38-133d-4526-9a9c-a14b9fd12bff None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:18.418433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 350/703] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Aug 1 11:51:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:18.423956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4d3ff-b89d-43e7-bb51-a5552d01b4ac None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:18.549179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4d3ff-b89d-43e7-bb51-a5552d01b4ac None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:18.582014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:18.586137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4e957-e4f1-47a6-98b5-aabd2a8bff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:18.586776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4e957-e4f1-47a6-98b5-aabd2a8bff40 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:18.587053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4e957-e4f1-47a6-98b5-aabd2a8bff40 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:18.587328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4e957-e4f1-47a6-98b5-aabd2a8bff40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:18.662661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4e957-e4f1-47a6-98b5-aabd2a8bff40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:18.664132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4e957-e4f1-47a6-98b5-aabd2a8bff40 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:18.765932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 354/704] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:51:18.771606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce14c38-133d-4526-9a9c-a14b9fd12bff None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:18.810073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33db269-175c-4a88-85ac-7d030618e0e8 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:18.810409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33db269-175c-4a88-85ac-7d030618e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:18.810619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33db269-175c-4a88-85ac-7d030618e0e8 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:18.812194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 355/705] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:18.817672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dc179-5c96-48cc-a6cb-58af0af6eb29 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:18.817914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dc179-5c96-48cc-a6cb-58af0af6eb29 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:18.818175 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175dc179-5c96-48cc-a6cb-58af0af6eb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:18.839322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-175dc179-5c96-48cc-a6cb-58af0af6eb29 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:18.841558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-175dc179-5c96-48cc-a6cb-58af0af6eb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:18.864197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:18.884454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:18.921050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce14c38-133d-4526-9a9c-a14b9fd12bff None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:18.940040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 356/706] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 01 11:51:18.962972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35bd82-e96c-41dd-96ba-a2b3fe68395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:18.963383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175dc179-5c96-48cc-a6cb-58af0af6eb29 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:18.963629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35bd82-e96c-41dd-96ba-a2b3fe68395a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:18.963905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35bd82-e96c-41dd-96ba-a2b3fe68395a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:18.964236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35bd82-e96c-41dd-96ba-a2b3fe68395a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:18.975332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35bd82-e96c-41dd-96ba-a2b3fe68395a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:18.976811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35bd82-e96c-41dd-96ba-a2b3fe68395a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-de35bd82-e96c-41dd-96ba-a2b3fe68395a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:18.993023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:18.993964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:18.995082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:18.995082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:18.995082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:18.995082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:18.995082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 351/707] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:19.001205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35bd82-e96c-41dd-96ba-a2b3fe68395a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:19.071719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175dc179-5c96-48cc-a6cb-58af0af6eb29 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:19.108767 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa576ae-2e90-4f2d-8d78-a4ea35651213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:19.109476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa576ae-2e90-4f2d-8d78-a4ea35651213 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.110121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa576ae-2e90-4f2d-8d78-a4ea35651213 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.110390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa576ae-2e90-4f2d-8d78-a4ea35651213 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.111026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa576ae-2e90-4f2d-8d78-a4ea35651213 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:19.112460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa576ae-2e90-4f2d-8d78-a4ea35651213 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-faa576ae-2e90-4f2d-8d78-a4ea35651213 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:19.124168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:19.126575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:19.128572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 357/708] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Aug 1 11:51:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:19.130708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7564-5131-4212-aeb5-09c84b2a7b16 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.130708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7564-5131-4212-aeb5-09c84b2a7b16 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.130708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e7564-5131-4212-aeb5-09c84b2a7b16 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.133764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 358/709] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:19.134540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e7564-5131-4212-aeb5-09c84b2a7b16 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:19.163323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35bd82-e96c-41dd-96ba-a2b3fe68395a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:19.171609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:19.203095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8350bcb3-4874-48e9-815c-59a4fd670224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:19.204140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350bcb3-4874-48e9-815c-59a4fd670224 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.204647 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350bcb3-4874-48e9-815c-59a4fd670224 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.205215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8350bcb3-4874-48e9-815c-59a4fd670224 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.206232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350bcb3-4874-48e9-815c-59a4fd670224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:19.206740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350bcb3-4874-48e9-815c-59a4fd670224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:19.212457 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:19.213641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8350bcb3-4874-48e9-815c-59a4fd670224 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:19.229142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 352/710] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:19.230896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f39b-87d5-4ed7-a16f-52510ee9b170 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.231378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f39b-87d5-4ed7-a16f-52510ee9b170 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.231855 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09f39b-87d5-4ed7-a16f-52510ee9b170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.251499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ab09f39b-87d5-4ed7-a16f-52510ee9b170 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:19.253253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ab09f39b-87d5-4ed7-a16f-52510ee9b170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:19.255709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e7564-5131-4212-aeb5-09c84b2a7b16 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:19.290782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b884b-c124-4377-b98c-43864b213c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:19.291457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b884b-c124-4377-b98c-43864b213c6e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.291736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b884b-c124-4377-b98c-43864b213c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.292001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b884b-c124-4377-b98c-43864b213c6e None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.292741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b884b-c124-4377-b98c-43864b213c6e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:19.293041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b884b-c124-4377-b98c-43864b213c6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:19.294552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b884b-c124-4377-b98c-43864b213c6e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:19.303817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 359/711] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Aug 1 11:51:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:19.303817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b884b-c124-4377-b98c-43864b213c6e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:19.334624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 353/712] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:19.335748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09f39b-87d5-4ed7-a16f-52510ee9b170 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:19.426014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:19.436771 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b884b-c124-4377-b98c-43864b213c6e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:19.444864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09f39b-87d5-4ed7-a16f-52510ee9b170 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:19.453262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:19.473585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a67db7-daf2-4594-8508-ed1e196732de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:19.474328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a67db7-daf2-4594-8508-ed1e196732de None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.474701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a67db7-daf2-4594-8508-ed1e196732de None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.475068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a67db7-daf2-4594-8508-ed1e196732de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.483272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a67db7-daf2-4594-8508-ed1e196732de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:19.485224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69471850-19e7-4b6a-b238-1fae1c04598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:19.485514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a67db7-daf2-4594-8508-ed1e196732de None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:19.486223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69471850-19e7-4b6a-b238-1fae1c04598b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.486650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69471850-19e7-4b6a-b238-1fae1c04598b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.487063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69471850-19e7-4b6a-b238-1fae1c04598b None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.488006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69471850-19e7-4b6a-b238-1fae1c04598b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:19.488576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69471850-19e7-4b6a-b238-1fae1c04598b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:19.490253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69471850-19e7-4b6a-b238-1fae1c04598b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-b8a67db7-daf2-4594-8508-ed1e196732de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:19.504321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:19.505304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:19.506277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:19.506277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:19.506277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:19.506277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:19.506277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:19.506277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:19.506277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:19.506277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:19.506963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 354/714] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Aug 1 11:51:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:19.507416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 360/714] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:19.508363 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69471850-19e7-4b6a-b238-1fae1c04598b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:19.508594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a67db7-daf2-4594-8508-ed1e196732de None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:19.623005 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a67db7-daf2-4594-8508-ed1e196732de None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:19.666264 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f028d3-d189-4c87-8b9a-3fe7cc6a26b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:19.666908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f028d3-d189-4c87-8b9a-3fe7cc6a26b1 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.667182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f028d3-d189-4c87-8b9a-3fe7cc6a26b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.667451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f028d3-d189-4c87-8b9a-3fe7cc6a26b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.669880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f028d3-d189-4c87-8b9a-3fe7cc6a26b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:19.670198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f028d3-d189-4c87-8b9a-3fe7cc6a26b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:19.671685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f028d3-d189-4c87-8b9a-3fe7cc6a26b1 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:19.698696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 361/715] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:19.704922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69471850-19e7-4b6a-b238-1fae1c04598b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:19.708717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f028d3-d189-4c87-8b9a-3fe7cc6a26b1 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:19.746337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:19.790149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f8ba2-164c-43c8-a219-88f4152f193b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:19.791197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f8ba2-164c-43c8-a219-88f4152f193b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.791677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f8ba2-164c-43c8-a219-88f4152f193b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.792222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f8ba2-164c-43c8-a219-88f4152f193b None admin] PATH_INFO: `/v3/services/network` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.793100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f8ba2-164c-43c8-a219-88f4152f193b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:19.794764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f8ba2-164c-43c8-a219-88f4152f193b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-b28f8ba2-164c-43c8-a219-88f4152f193b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:19.807866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:19.808981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 11:51:19.809853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:19.812875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 355/716] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:19 2022] GET /identity/v3/services/network => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:19.819443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f8ba2-164c-43c8-a219-88f4152f193b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:19.860239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:19.871519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f028d3-d189-4c87-8b9a-3fe7cc6a26b1 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:19.915625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c7230-f926-4624-8f58-f75903210c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:19.916468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c7230-f926-4624-8f58-f75903210c10 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.920322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c7230-f926-4624-8f58-f75903210c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.920322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c7230-f926-4624-8f58-f75903210c10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.920322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c7230-f926-4624-8f58-f75903210c10 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=db3ff1b21c784e8cb5bffcb79f230d08) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:19.920322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c7230-f926-4624-8f58-f75903210c10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:19.920694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c7230-f926-4624-8f58-f75903210c10 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:19.927312 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 362/717] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:19 2022] GET /identity/v3/role_assignments?user.id=db3ff1b21c784e8cb5bffcb79f230d08&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:19.943753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f8ba2-164c-43c8-a219-88f4152f193b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:19.983386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c97061-2f87-4cc7-96ff-3606da0ec726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:19.984038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c97061-2f87-4cc7-96ff-3606da0ec726 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:19.984338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c97061-2f87-4cc7-96ff-3606da0ec726 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:19.984973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c97061-2f87-4cc7-96ff-3606da0ec726 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:19.985679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c97061-2f87-4cc7-96ff-3606da0ec726 None admin] RBAC: Adding query filter params (name=network) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:19.986086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c97061-2f87-4cc7-96ff-3606da0ec726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:19.987522 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c97061-2f87-4cc7-96ff-3606da0ec726 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:19.993506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 356/718] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:19.999204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c7230-f926-4624-8f58-f75903210c10 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:20.080750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:20.108055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c7230-f926-4624-8f58-f75903210c10 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:20.144484 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6318742d-83da-49a4-a231-796e5650039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:20.145122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318742d-83da-49a4-a231-796e5650039f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:20.145401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318742d-83da-49a4-a231-796e5650039f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:20.145672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6318742d-83da-49a4-a231-796e5650039f None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:20.146362 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318742d-83da-49a4-a231-796e5650039f None admin] RBAC: Adding query filter params (type=network) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:20.146672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318742d-83da-49a4-a231-796e5650039f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:20.148065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6318742d-83da-49a4-a231-796e5650039f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:20.154785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 363/719] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:20.599323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee3ce3-2168-407b-bd2c-b9708931a12a None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:20.599886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee3ce3-2168-407b-bd2c-b9708931a12a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:20.599886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee3ce3-2168-407b-bd2c-b9708931a12a None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:20.601561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 357/720] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:20.606832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c90e4-e224-4a5f-aa82-b7dacfcfd98d None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:20.607123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c90e4-e224-4a5f-aa82-b7dacfcfd98d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:20.607400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c90e4-e224-4a5f-aa82-b7dacfcfd98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:20.626102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c41c90e4-e224-4a5f-aa82-b7dacfcfd98d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:20.627296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c41c90e4-e224-4a5f-aa82-b7dacfcfd98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:20.706573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 364/721] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:20.742388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee3ce3-2168-407b-bd2c-b9708931a12a None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:20.866791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:20.874208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee3ce3-2168-407b-bd2c-b9708931a12a None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:20.922214 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f773031-a3df-445d-90a1-e116b14f0fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:20.922873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f773031-a3df-445d-90a1-e116b14f0fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:20.923147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f773031-a3df-445d-90a1-e116b14f0fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:20.923417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f773031-a3df-445d-90a1-e116b14f0fa8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:20.940032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f773031-a3df-445d-90a1-e116b14f0fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-3f773031-a3df-445d-90a1-e116b14f0fa8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:20.945226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:20.946150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:20.946964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 358/722] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:20.952613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c90e4-e224-4a5f-aa82-b7dacfcfd98d None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:21.022992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:21.031941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f6909-f11b-47a7-9d35-7cdfd7e4b9f7 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:21.032257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f6909-f11b-47a7-9d35-7cdfd7e4b9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:21.032522 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f6909-f11b-47a7-9d35-7cdfd7e4b9f7 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:21.033881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 359/723] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:21.040574 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7bb5d-d3f7-4295-bb01-2bccc42829ca None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:21.040853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7bb5d-d3f7-4295-bb01-2bccc42829ca None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:21.041128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7bb5d-d3f7-4295-bb01-2bccc42829ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:21.061431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b6a7bb5d-d3f7-4295-bb01-2bccc42829ca None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:21.062934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b6a7bb5d-d3f7-4295-bb01-2bccc42829ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:21.105308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c90e4-e224-4a5f-aa82-b7dacfcfd98d None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:21.168660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 360/724] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:21.172788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6510ded-5b2e-4d66-906d-84c01f3f6f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:21.173632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6510ded-5b2e-4d66-906d-84c01f3f6f93 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:21.174123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6510ded-5b2e-4d66-906d-84c01f3f6f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:21.174573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6510ded-5b2e-4d66-906d-84c01f3f6f93 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:21.175483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6510ded-5b2e-4d66-906d-84c01f3f6f93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:21.176051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6510ded-5b2e-4d66-906d-84c01f3f6f93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:21.177646 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6510ded-5b2e-4d66-906d-84c01f3f6f93 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:21.191880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 365/725] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:21.197066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7bb5d-d3f7-4295-bb01-2bccc42829ca None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:21.214464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6510ded-5b2e-4d66-906d-84c01f3f6f93 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:21.353828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:21.376583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7bb5d-d3f7-4295-bb01-2bccc42829ca None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:21.392480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6510ded-5b2e-4d66-906d-84c01f3f6f93 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:21.436537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d83783-de15-4abb-b9f0-d8a615b4a105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:21.437109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d83783-de15-4abb-b9f0-d8a615b4a105 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:21.437399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d83783-de15-4abb-b9f0-d8a615b4a105 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:21.437675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d83783-de15-4abb-b9f0-d8a615b4a105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:21.452559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c57e2-2733-4bed-b10d-80504807b6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:21.453227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c57e2-2733-4bed-b10d-80504807b6eb None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:21.453496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c57e2-2733-4bed-b10d-80504807b6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:21.453771 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c57e2-2733-4bed-b10d-80504807b6eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:21.459664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d83783-de15-4abb-b9f0-d8a615b4a105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:21.460603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:21.461173 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d83783-de15-4abb-b9f0-d8a615b4a105 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:21.478346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c57e2-2733-4bed-b10d-80504807b6eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-c27c57e2-2733-4bed-b10d-80504807b6eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:21.483795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:21.485096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:21.485939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:21.488385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 366/726] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-e1d83783-de15-4abb-b9f0-d8a615b4a105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:21.495925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:21.496825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:21.498761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:21.498761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:21.498761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:21.498761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:21.498761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 361/727] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:21.499383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c57e2-2733-4bed-b10d-80504807b6eb None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:21.502665 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d83783-de15-4abb-b9f0-d8a615b4a105 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:21.651424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d83783-de15-4abb-b9f0-d8a615b4a105 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:21.685452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c57e2-2733-4bed-b10d-80504807b6eb None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:21.706780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42ade1-e604-48f8-849a-8dbcb13e18d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:21.707348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42ade1-e604-48f8-849a-8dbcb13e18d6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:21.707632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42ade1-e604-48f8-849a-8dbcb13e18d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:21.708006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42ade1-e604-48f8-849a-8dbcb13e18d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:21.708945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42ade1-e604-48f8-849a-8dbcb13e18d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:21.709259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42ade1-e604-48f8-849a-8dbcb13e18d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:21.710654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42ade1-e604-48f8-849a-8dbcb13e18d6 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:21.724705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeedc6c-f94f-41ff-b924-ff6692756161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:21.725417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeedc6c-f94f-41ff-b924-ff6692756161 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:21.725694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeedc6c-f94f-41ff-b924-ff6692756161 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:21.725969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeedc6c-f94f-41ff-b924-ff6692756161 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:21.726750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeedc6c-f94f-41ff-b924-ff6692756161 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:21.727100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeedc6c-f94f-41ff-b924-ff6692756161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:21.728539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeedc6c-f94f-41ff-b924-ff6692756161 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:21.736992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:21.742945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 362/728] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:21.748990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 367/729] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:21.749956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42ade1-e604-48f8-849a-8dbcb13e18d6 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:21.757799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeedc6c-f94f-41ff-b924-ff6692756161 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:21.820281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:21.899555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeedc6c-f94f-41ff-b924-ff6692756161 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:21.928411 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42ade1-e604-48f8-849a-8dbcb13e18d6 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:21.962558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd3f976-236b-4ffb-b1e3-004a98a651a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:21.963214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3f976-236b-4ffb-b1e3-004a98a651a5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:21.963404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3f976-236b-4ffb-b1e3-004a98a651a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:21.963676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd3f976-236b-4ffb-b1e3-004a98a651a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:21.966790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f206b27-e523-4ca6-b2c9-9430723a2448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:21.967437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f206b27-e523-4ca6-b2c9-9430723a2448 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:21.967714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f206b27-e523-4ca6-b2c9-9430723a2448 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:21.967983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f206b27-e523-4ca6-b2c9-9430723a2448 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:21.981568 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f206b27-e523-4ca6-b2c9-9430723a2448 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:21.983023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f206b27-e523-4ca6-b2c9-9430723a2448 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-1f206b27-e523-4ca6-b2c9-9430723a2448 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:22.006483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:22.008699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:22.010949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 363/730] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Aug 1 11:51:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:22.025565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f206b27-e523-4ca6-b2c9-9430723a2448 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:22.125008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:22.132605 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3f976-236b-4ffb-b1e3-004a98a651a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:22.136837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd3f976-236b-4ffb-b1e3-004a98a651a5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-3fd3f976-236b-4ffb-b1e3-004a98a651a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:22.162952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:22.164659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:22.165903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:22.165903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:22.165903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:22.165903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:22.165903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 368/731] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:22.168697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd1e8e-7b3a-4a00-8889-ec7fbe14d907 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.170152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd1e8e-7b3a-4a00-8889-ec7fbe14d907 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.170152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd1e8e-7b3a-4a00-8889-ec7fbe14d907 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.170684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 369/732] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:22.171389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd1e8e-7b3a-4a00-8889-ec7fbe14d907 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:22.182176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f206b27-e523-4ca6-b2c9-9430723a2448 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:22.229285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ff1d2-bab8-48e9-899d-5603cd346209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:22.230042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ff1d2-bab8-48e9-899d-5603cd346209 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.230411 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ff1d2-bab8-48e9-899d-5603cd346209 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.231320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ff1d2-bab8-48e9-899d-5603cd346209 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.232310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ff1d2-bab8-48e9-899d-5603cd346209 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:22.233139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ff1d2-bab8-48e9-899d-5603cd346209 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:22.235464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ff1d2-bab8-48e9-899d-5603cd346209 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:22.260156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 364/733] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Aug 1 11:51:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:22.261977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc5544-c162-448a-a33a-7d7ef90147bd None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.262263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc5544-c162-448a-a33a-7d7ef90147bd None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.262536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc5544-c162-448a-a33a-7d7ef90147bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.285526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-befc5544-c162-448a-a33a-7d7ef90147bd None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:22.286726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-befc5544-c162-448a-a33a-7d7ef90147bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:22.304044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd1e8e-7b3a-4a00-8889-ec7fbe14d907 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:22.307227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:22.343991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d335ac7-a670-4de8-8d37-0878f55ad526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:22.345062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d335ac7-a670-4de8-8d37-0878f55ad526 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.345559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d335ac7-a670-4de8-8d37-0878f55ad526 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.345961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d335ac7-a670-4de8-8d37-0878f55ad526 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.346829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d335ac7-a670-4de8-8d37-0878f55ad526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:22.347268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d335ac7-a670-4de8-8d37-0878f55ad526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:22.349151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d335ac7-a670-4de8-8d37-0878f55ad526 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:22.368447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 365/734] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:22.368447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc5544-c162-448a-a33a-7d7ef90147bd None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:22.371527 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 370/735] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:22.377581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d335ac7-a670-4de8-8d37-0878f55ad526 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:22.414200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:22.465792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc5544-c162-448a-a33a-7d7ef90147bd None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:22.503833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8705f5-a5db-490c-b01b-6a3cde209b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:22.504834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8705f5-a5db-490c-b01b-6a3cde209b8d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.505315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8705f5-a5db-490c-b01b-6a3cde209b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.505788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8705f5-a5db-490c-b01b-6a3cde209b8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.506355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d335ac7-a670-4de8-8d37-0878f55ad526 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:22.543404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf3de9-04be-4121-9852-23eb86b1a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:22.544381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf3de9-04be-4121-9852-23eb86b1a6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.544661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf3de9-04be-4121-9852-23eb86b1a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.544928 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf3de9-04be-4121-9852-23eb86b1a6c7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.552694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf3de9-04be-4121-9852-23eb86b1a6c7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:22.554269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf3de9-04be-4121-9852-23eb86b1a6c7 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-baaf3de9-04be-4121-9852-23eb86b1a6c7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:22.566208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:22.567177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:22.568143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:22.568143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:22.568143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:22.568143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:22.568143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:22.568143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:22.568143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:22.568143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:22.568143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:22.568143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:22.572144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 371/736] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:22.573013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf3de9-04be-4121-9852-23eb86b1a6c7 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:22.579529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8705f5-a5db-490c-b01b-6a3cde209b8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:22.581081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8705f5-a5db-490c-b01b-6a3cde209b8d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:22.599292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:22.635114 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:22.682148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf3de9-04be-4121-9852-23eb86b1a6c7 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:22.697077 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 366/737] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:51:22.698557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfcbdc-901c-4527-8a79-ecdfc45217c2 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.698841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfcbdc-901c-4527-8a79-ecdfc45217c2 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.699115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfcbdc-901c-4527-8a79-ecdfc45217c2 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.704350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 367/738] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:22.704350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfcbdc-901c-4527-8a79-ecdfc45217c2 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:22.725389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ffe06-3a86-4dd8-85b9-2cd8a0076b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:22.726014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ffe06-3a86-4dd8-85b9-2cd8a0076b6f None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.726318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ffe06-3a86-4dd8-85b9-2cd8a0076b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.726566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ffe06-3a86-4dd8-85b9-2cd8a0076b6f None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.727257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ffe06-3a86-4dd8-85b9-2cd8a0076b6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:22.728650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ffe06-3a86-4dd8-85b9-2cd8a0076b6f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:22.745886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 372/739] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:51:22 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:22.746574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ffe06-3a86-4dd8-85b9-2cd8a0076b6f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:22.840498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfcbdc-901c-4527-8a79-ecdfc45217c2 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:22.846156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:22.865448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ffe06-3a86-4dd8-85b9-2cd8a0076b6f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:22.884918 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a43245c-04b1-465a-ab78-de62b6b4fdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:22.885551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43245c-04b1-465a-ab78-de62b6b4fdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.885838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43245c-04b1-465a-ab78-de62b6b4fdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.886108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a43245c-04b1-465a-ab78-de62b6b4fdf4 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.886925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43245c-04b1-465a-ab78-de62b6b4fdf4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:22.887229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43245c-04b1-465a-ab78-de62b6b4fdf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:22.888780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a43245c-04b1-465a-ab78-de62b6b4fdf4 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:22.908098 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ef71d-cf9e-4df2-a73a-b7360c048a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:22.909014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef71d-cf9e-4df2-a73a-b7360c048a33 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.909423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef71d-cf9e-4df2-a73a-b7360c048a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.909699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef71d-cf9e-4df2-a73a-b7360c048a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.911465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 368/740] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Aug 1 11:51:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:22.912793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1017489-f743-4fc9-80a9-f5f0e323e423 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.913074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1017489-f743-4fc9-80a9-f5f0e323e423 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.913339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1017489-f743-4fc9-80a9-f5f0e323e423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.919836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:22.923764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ef71d-cf9e-4df2-a73a-b7360c048a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:22.925205 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ef71d-cf9e-4df2-a73a-b7360c048a33 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:22.934717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c1017489-f743-4fc9-80a9-f5f0e323e423 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:22.935157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c1017489-f743-4fc9-80a9-f5f0e323e423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-710ef71d-cf9e-4df2-a73a-b7360c048a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:22.943004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:22.943945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:22.944921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:22.944921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:22.944921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:22.944921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:22.944921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 373/741] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:22.946217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2181093-a1d7-4268-914a-39c0d64b7dbe None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:22.946507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2181093-a1d7-4268-914a-39c0d64b7dbe None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:22.946785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2181093-a1d7-4268-914a-39c0d64b7dbe None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:22.949012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 374/742] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:22.949189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2181093-a1d7-4268-914a-39c0d64b7dbe None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:23.007038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 369/743] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:23.007896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1017489-f743-4fc9-80a9-f5f0e323e423 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:23.064606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2181093-a1d7-4268-914a-39c0d64b7dbe None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:23.101835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcca3a-52d8-4717-8094-8f41bfed0b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:23.102369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcca3a-52d8-4717-8094-8f41bfed0b68 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:23.102647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcca3a-52d8-4717-8094-8f41bfed0b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:23.102921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcca3a-52d8-4717-8094-8f41bfed0b68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:23.106793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:23.134092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1017489-f743-4fc9-80a9-f5f0e323e423 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:23.164336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:23.171436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78587491-6727-425a-bd57-03ee3feb874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:23.172081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78587491-6727-425a-bd57-03ee3feb874a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:23.172511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78587491-6727-425a-bd57-03ee3feb874a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:23.172822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78587491-6727-425a-bd57-03ee3feb874a None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:23.173815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78587491-6727-425a-bd57-03ee3feb874a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:23.174265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78587491-6727-425a-bd57-03ee3feb874a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:23.175859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78587491-6727-425a-bd57-03ee3feb874a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:23.180346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcca3a-52d8-4717-8094-8f41bfed0b68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:23.181839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fcca3a-52d8-4717-8094-8f41bfed0b68 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:23.192465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 370/744] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:23.194179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe601d-fe55-436a-a07f-8322fe8283dc None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:23.194779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe601d-fe55-436a-a07f-8322fe8283dc None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:23.194883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe601d-fe55-436a-a07f-8322fe8283dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:23.213035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-dfbe601d-fe55-436a-a07f-8322fe8283dc None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:23.214238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-dfbe601d-fe55-436a-a07f-8322fe8283dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:23.288467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 371/745] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:23 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 01 11:51:23.288467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe601d-fe55-436a-a07f-8322fe8283dc None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:23.300512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 375/746] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:22 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 01 11:51:23.301405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcca3a-52d8-4717-8094-8f41bfed0b68 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:23.336019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:23.401445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe601d-fe55-436a-a07f-8322fe8283dc None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:23.405596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcca3a-52d8-4717-8094-8f41bfed0b68 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:23.430420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:23.445315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993abaa-7b11-4ff5-89f6-2e6cb41fb32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:23.446314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae5d64-fb17-4019-8534-6596dec25ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:23.446523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993abaa-7b11-4ff5-89f6-2e6cb41fb32e None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:23.446947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993abaa-7b11-4ff5-89f6-2e6cb41fb32e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:23.447085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae5d64-fb17-4019-8534-6596dec25ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:23.447395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae5d64-fb17-4019-8534-6596dec25ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:23.447697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993abaa-7b11-4ff5-89f6-2e6cb41fb32e None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:23.447868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae5d64-fb17-4019-8534-6596dec25ba8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:23.448788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993abaa-7b11-4ff5-89f6-2e6cb41fb32e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:23.450436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993abaa-7b11-4ff5-89f6-2e6cb41fb32e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:23.456386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae5d64-fb17-4019-8534-6596dec25ba8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:23.458078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae5d64-fb17-4019-8534-6596dec25ba8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:23.470041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 372/747] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:51:23 2022] POST /identity/v3/services => generated 240 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:23.471163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993abaa-7b11-4ff5-89f6-2e6cb41fb32e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-48ae5d64-fb17-4019-8534-6596dec25ba8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:23.471290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:23.472094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:23.473031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:23.473031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:23.473031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:23.473031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:23.473031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:23.473031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:23.473031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:23.473031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:23.477388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 376/748] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:23.478399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae5d64-fb17-4019-8534-6596dec25ba8 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:23.599466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:23.610158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae5d64-fb17-4019-8534-6596dec25ba8 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:23.632348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993abaa-7b11-4ff5-89f6-2e6cb41fb32e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:23.658499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c37ff1-077d-494f-aa93-4d7a1b093c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:23.659081 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c37ff1-077d-494f-aa93-4d7a1b093c06 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:23.659369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c37ff1-077d-494f-aa93-4d7a1b093c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:23.659627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c37ff1-077d-494f-aa93-4d7a1b093c06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:23.673384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c37ff1-077d-494f-aa93-4d7a1b093c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:23.675717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0b9d5-f4e7-497b-bf70-053876352473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:23.676423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0b9d5-f4e7-497b-bf70-053876352473 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:23.676714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0b9d5-f4e7-497b-bf70-053876352473 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:23.676991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0b9d5-f4e7-497b-bf70-053876352473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-e1c37ff1-077d-494f-aa93-4d7a1b093c06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:23.678885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:23.679884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:23.680792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 377/749] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:23.681497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c37ff1-077d-494f-aa93-4d7a1b093c06 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:23.695547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0b9d5-f4e7-497b-bf70-053876352473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:23.697596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0b9d5-f4e7-497b-bf70-053876352473 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-04f0b9d5-f4e7-497b-bf70-053876352473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:23.717762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:23.718740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:23.719905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:23.719905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:23.719905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:23.719905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:23.719905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 373/750] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:23.720954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0b9d5-f4e7-497b-bf70-053876352473 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:23.759111 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:23.818791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c37ff1-077d-494f-aa93-4d7a1b093c06 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:23.861533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0b9d5-f4e7-497b-bf70-053876352473 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:23.871945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc903f89-66bf-47c8-b65f-d7ae17a70906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:23.872704 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc903f89-66bf-47c8-b65f-d7ae17a70906 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:23.873050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc903f89-66bf-47c8-b65f-d7ae17a70906 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:23.873414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc903f89-66bf-47c8-b65f-d7ae17a70906 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:23.874735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc903f89-66bf-47c8-b65f-d7ae17a70906 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:23.875109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc903f89-66bf-47c8-b65f-d7ae17a70906 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:23.877221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc903f89-66bf-47c8-b65f-d7ae17a70906 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:23.897309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 378/751] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:23.898849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc903f89-66bf-47c8-b65f-d7ae17a70906 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:23.908645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9f395-447d-42f8-9a4f-9b5995702f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:23.909596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9f395-447d-42f8-9a4f-9b5995702f5c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:23.910168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9f395-447d-42f8-9a4f-9b5995702f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:23.910651 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9f395-447d-42f8-9a4f-9b5995702f5c None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:23.912245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9f395-447d-42f8-9a4f-9b5995702f5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:23.912786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9f395-447d-42f8-9a4f-9b5995702f5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:23.914867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9f395-447d-42f8-9a4f-9b5995702f5c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:23.925300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:23.928389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 374/752] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:23.929520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9f395-447d-42f8-9a4f-9b5995702f5c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:24.036677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc903f89-66bf-47c8-b65f-d7ae17a70906 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:24.049502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:24.062979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9f395-447d-42f8-9a4f-9b5995702f5c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:24.085335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd8e56-11e0-4b08-8051-12478d64610b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:24.086637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd8e56-11e0-4b08-8051-12478d64610b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:24.087087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd8e56-11e0-4b08-8051-12478d64610b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:24.087565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd8e56-11e0-4b08-8051-12478d64610b None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:24.088498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd8e56-11e0-4b08-8051-12478d64610b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:24.089407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd8e56-11e0-4b08-8051-12478d64610b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:24.091187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd8e56-11e0-4b08-8051-12478d64610b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:24.115591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 379/753] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:24.116697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd8e56-11e0-4b08-8051-12478d64610b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:24.117428 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1318f-ce38-49f2-a192-91b67739167e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:24.118273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1318f-ce38-49f2-a192-91b67739167e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:24.118741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1318f-ce38-49f2-a192-91b67739167e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:24.119186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1318f-ce38-49f2-a192-91b67739167e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:24.120641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1318f-ce38-49f2-a192-91b67739167e None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=bf76a8ec03e444e7b42e2997b8f73808) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:24.122176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1318f-ce38-49f2-a192-91b67739167e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:24.123818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1318f-ce38-49f2-a192-91b67739167e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:24.132724 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 375/754] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:23 2022] GET /identity/v3/role_assignments?user.id=bf76a8ec03e444e7b42e2997b8f73808&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:24.134727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1318f-ce38-49f2-a192-91b67739167e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:24.218354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:24.267201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd8e56-11e0-4b08-8051-12478d64610b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:24.285763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1318f-ce38-49f2-a192-91b67739167e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:24.330316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fc263-023e-4782-a2aa-d6c0c225f0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:24.331568 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fc263-023e-4782-a2aa-d6c0c225f0b5 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:24.332087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fc263-023e-4782-a2aa-d6c0c225f0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:24.332578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fc263-023e-4782-a2aa-d6c0c225f0b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:24.340963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fc263-023e-4782-a2aa-d6c0c225f0b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:24.343371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fc263-023e-4782-a2aa-d6c0c225f0b5 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:24.356633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73755848-a2f5-4f29-92ae-9e7eddef26c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:24.357442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73755848-a2f5-4f29-92ae-9e7eddef26c9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:24.357738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73755848-a2f5-4f29-92ae-9e7eddef26c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:24.358024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73755848-a2f5-4f29-92ae-9e7eddef26c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-192fc263-023e-4782-a2aa-d6c0c225f0b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:24.359215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:24.360584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:24.361497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:24.361497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:24.361497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:24.361497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:24.361497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:24.361497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:24.361497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:24.361497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:24.365114 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 376/755] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:24.371629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fc263-023e-4782-a2aa-d6c0c225f0b5 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:24.393136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73755848-a2f5-4f29-92ae-9e7eddef26c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:24.394809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73755848-a2f5-4f29-92ae-9e7eddef26c9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:24.409053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-73755848-a2f5-4f29-92ae-9e7eddef26c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:24.422485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:24.423458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:24.424640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:24.424640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:24.424640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:24.424640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:24.424640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 380/756] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:24.432659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73755848-a2f5-4f29-92ae-9e7eddef26c9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:24.565548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fc263-023e-4782-a2aa-d6c0c225f0b5 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:24.635728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76f45d-e1f5-4842-82f8-d90b21636b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:24.636817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76f45d-e1f5-4842-82f8-d90b21636b07 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:24.637299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76f45d-e1f5-4842-82f8-d90b21636b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:24.637471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73755848-a2f5-4f29-92ae-9e7eddef26c9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:24.637938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76f45d-e1f5-4842-82f8-d90b21636b07 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:24.638911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76f45d-e1f5-4842-82f8-d90b21636b07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:24.639427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76f45d-e1f5-4842-82f8-d90b21636b07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:24.641098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76f45d-e1f5-4842-82f8-d90b21636b07 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:24.666380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:24.670759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 377/757] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:24.673905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4102740-3f70-43c5-ab08-d0b8dc5e3d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:24.674557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4102740-3f70-43c5-ab08-d0b8dc5e3d2e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:24.674847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4102740-3f70-43c5-ab08-d0b8dc5e3d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:24.675125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4102740-3f70-43c5-ab08-d0b8dc5e3d2e None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:24.675879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4102740-3f70-43c5-ab08-d0b8dc5e3d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:24.676186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4102740-3f70-43c5-ab08-d0b8dc5e3d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:24.677824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4102740-3f70-43c5-ab08-d0b8dc5e3d2e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:24.681464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76f45d-e1f5-4842-82f8-d90b21636b07 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:24.692680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 381/758] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:24.705462 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4102740-3f70-43c5-ab08-d0b8dc5e3d2e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:24.835422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4102740-3f70-43c5-ab08-d0b8dc5e3d2e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:24.849034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:24.890435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76f45d-e1f5-4842-82f8-d90b21636b07 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:24.910764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d76c3e-e9f0-4d61-81bb-fb39f9210ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:24.911426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d76c3e-e9f0-4d61-81bb-fb39f9210ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:24.911700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d76c3e-e9f0-4d61-81bb-fb39f9210ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:24.912014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d76c3e-e9f0-4d61-81bb-fb39f9210ef0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:24.922557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d76c3e-e9f0-4d61-81bb-fb39f9210ef0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:24.924933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d76c3e-e9f0-4d61-81bb-fb39f9210ef0 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:24.938936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bbd99f-d58d-4627-8f99-5595bf8876c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:24.939758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd99f-d58d-4627-8f99-5595bf8876c0 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:24.940104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd99f-d58d-4627-8f99-5595bf8876c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:24.940467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bbd99f-d58d-4627-8f99-5595bf8876c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-b9d76c3e-e9f0-4d61-81bb-fb39f9210ef0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:24.941328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:24.942243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:24.943182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:24.943182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:24.943182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:24.943182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:24.943182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:24.943182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:24.943182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:24.943182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:24.943182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:24.943182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:24.943718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbd99f-d58d-4627-8f99-5595bf8876c0 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=b9b6a96421e349a88a38590cfce167fb) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:24.943718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbd99f-d58d-4627-8f99-5595bf8876c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:24.943854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 382/759] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Aug 1 11:51:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:24.943921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bbd99f-d58d-4627-8f99-5595bf8876c0 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:24.943988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56593b-6712-4758-a31f-1d6ca3f68567 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:24.944535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56593b-6712-4758-a31f-1d6ca3f68567 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:24.944535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56593b-6712-4758-a31f-1d6ca3f68567 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:24.948313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 383/760] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:24.950713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56593b-6712-4758-a31f-1d6ca3f68567 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:24.955190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 378/761] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:24 2022] GET /identity/v3/role_assignments?user.id=b9b6a96421e349a88a38590cfce167fb&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:24.956348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f338a-ca85-487f-8bc5-b52cccc1ddd7 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:24.956763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f338a-ca85-487f-8bc5-b52cccc1ddd7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:24.957042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f338a-ca85-487f-8bc5-b52cccc1ddd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:24.977716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c07f338a-ca85-487f-8bc5-b52cccc1ddd7 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:24.979073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c07f338a-ca85-487f-8bc5-b52cccc1ddd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:25.016811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:25.071147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 379/762] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:25.098737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f338a-ca85-487f-8bc5-b52cccc1ddd7 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:25.109911 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56593b-6712-4758-a31f-1d6ca3f68567 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:25.166160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2468463-f106-46ef-a9a3-4077a126910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:25.166826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2468463-f106-46ef-a9a3-4077a126910c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:25.167110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2468463-f106-46ef-a9a3-4077a126910c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:25.167385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2468463-f106-46ef-a9a3-4077a126910c None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:25.168383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2468463-f106-46ef-a9a3-4077a126910c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:25.168712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2468463-f106-46ef-a9a3-4077a126910c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:25.171121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2468463-f106-46ef-a9a3-4077a126910c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:25.194196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 384/763] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Aug 1 11:51:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:25.201557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2468463-f106-46ef-a9a3-4077a126910c None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:25.202412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:25.236333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f338a-ca85-487f-8bc5-b52cccc1ddd7 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:25.277149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84005d65-5e55-4ca4-8bb3-e6b0d57087b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:25.277942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005d65-5e55-4ca4-8bb3-e6b0d57087b2 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:25.278285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005d65-5e55-4ca4-8bb3-e6b0d57087b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:25.278621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005d65-5e55-4ca4-8bb3-e6b0d57087b2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:25.279409 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84005d65-5e55-4ca4-8bb3-e6b0d57087b2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:25.281772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84005d65-5e55-4ca4-8bb3-e6b0d57087b2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-84005d65-5e55-4ca4-8bb3-e6b0d57087b2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:25.292582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:25.293611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:25.294793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 380/764] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Aug 1 11:51:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:25.304345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84005d65-5e55-4ca4-8bb3-e6b0d57087b2 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:25.430474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84005d65-5e55-4ca4-8bb3-e6b0d57087b2 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:25.442724 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2468463-f106-46ef-a9a3-4077a126910c None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:25.450841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:25.472914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea90e0d-7fc9-47e2-9575-145a85c6511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:25.473746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea90e0d-7fc9-47e2-9575-145a85c6511e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:25.474121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea90e0d-7fc9-47e2-9575-145a85c6511e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:25.474491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea90e0d-7fc9-47e2-9575-145a85c6511e None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:25.475350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea90e0d-7fc9-47e2-9575-145a85c6511e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:25.475772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea90e0d-7fc9-47e2-9575-145a85c6511e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:25.477356 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea90e0d-7fc9-47e2-9575-145a85c6511e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:25.482775 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:25.485892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 381/765] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Aug 1 11:51:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:25.493341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea90e0d-7fc9-47e2-9575-145a85c6511e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:25.534334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceac742-51f0-44dd-8cdc-5b83aa0ebfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:25.535318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac742-51f0-44dd-8cdc-5b83aa0ebfbb None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:25.535746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac742-51f0-44dd-8cdc-5b83aa0ebfbb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:25.536162 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceac742-51f0-44dd-8cdc-5b83aa0ebfbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:25.556877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceac742-51f0-44dd-8cdc-5b83aa0ebfbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:25.558626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceac742-51f0-44dd-8cdc-5b83aa0ebfbb None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-7ceac742-51f0-44dd-8cdc-5b83aa0ebfbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:25.574922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:25.575886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:25.576811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:25.576811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:25.576811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:25.576811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:25.579132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 385/766] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:25.593414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceac742-51f0-44dd-8cdc-5b83aa0ebfbb None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:25.629849 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea90e0d-7fc9-47e2-9575-145a85c6511e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:25.676082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38e4f6-8d0e-4ef0-85e0-8529069c8ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:25.676759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38e4f6-8d0e-4ef0-85e0-8529069c8ebc None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:25.677034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38e4f6-8d0e-4ef0-85e0-8529069c8ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:25.677302 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38e4f6-8d0e-4ef0-85e0-8529069c8ebc None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:25.677985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38e4f6-8d0e-4ef0-85e0-8529069c8ebc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:25.678285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38e4f6-8d0e-4ef0-85e0-8529069c8ebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:25.681663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38e4f6-8d0e-4ef0-85e0-8529069c8ebc None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:25.702669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 382/767] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Aug 1 11:51:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:25.713388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38e4f6-8d0e-4ef0-85e0-8529069c8ebc None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:25.796534 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceac742-51f0-44dd-8cdc-5b83aa0ebfbb None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:25.836275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:25.885967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38e4f6-8d0e-4ef0-85e0-8529069c8ebc None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:25.925431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf30510-2c04-4d52-b07b-cc8aba3f9f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:25.926329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf30510-2c04-4d52-b07b-cc8aba3f9f6d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:25.926812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf30510-2c04-4d52-b07b-cc8aba3f9f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:25.927222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf30510-2c04-4d52-b07b-cc8aba3f9f6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:25.928161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf30510-2c04-4d52-b07b-cc8aba3f9f6d None admin] RBAC: Adding query filter params (interface=public, service_id=2688a6c09ab24ff78a707042c2291d5d, region_id=RegionOne) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:25.928635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf30510-2c04-4d52-b07b-cc8aba3f9f6d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:25.929024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9e981-9e3e-422b-8d95-9619f196c0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:25.929680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9e981-9e3e-422b-8d95-9619f196c0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:25.929964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9e981-9e3e-422b-8d95-9619f196c0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:25.930239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9e981-9e3e-422b-8d95-9619f196c0e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:25.930913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf30510-2c04-4d52-b07b-cc8aba3f9f6d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:25.931023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9e981-9e3e-422b-8d95-9619f196c0e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:25.931439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9e981-9e3e-422b-8d95-9619f196c0e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:25.933961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9e981-9e3e-422b-8d95-9619f196c0e5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:25.944431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 383/768] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Aug 1 11:51:25 2022] GET /identity/v3/endpoints?service_id=2688a6c09ab24ff78a707042c2291d5d&interface=public®ion_id=RegionOne => generated 194 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:25.946590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf30510-2c04-4d52-b07b-cc8aba3f9f6d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:25.964277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 386/769] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:25.980598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9e981-9e3e-422b-8d95-9619f196c0e5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:26.042472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:26.070171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf30510-2c04-4d52-b07b-cc8aba3f9f6d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:26.098480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:26.108369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519f756-a5e7-4ef3-a13c-efdb49955528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:26.109209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519f756-a5e7-4ef3-a13c-efdb49955528 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.109972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519f756-a5e7-4ef3-a13c-efdb49955528 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.110387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519f756-a5e7-4ef3-a13c-efdb49955528 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.111205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519f756-a5e7-4ef3-a13c-efdb49955528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:26.112827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519f756-a5e7-4ef3-a13c-efdb49955528 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:26.119056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 384/770] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Aug 1 11:51:25 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:26.197391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a0502-21cb-48b8-b97f-e5b8265a909e None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.197892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a0502-21cb-48b8-b97f-e5b8265a909e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.198324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a0502-21cb-48b8-b97f-e5b8265a909e None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.200065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 385/771] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:51:26.205161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809232a-5daa-4ec8-819b-439de4cc5637 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.205638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809232a-5daa-4ec8-819b-439de4cc5637 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.206054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809232a-5daa-4ec8-819b-439de4cc5637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.244851 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9e981-9e3e-422b-8d95-9619f196c0e5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:26.256444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-7809232a-5daa-4ec8-819b-439de4cc5637 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:26.257688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7809232a-5daa-4ec8-819b-439de4cc5637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:26.290493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c7101-7371-4d92-b798-c265b6b7621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:26.291170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c7101-7371-4d92-b798-c265b6b7621f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.291447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c7101-7371-4d92-b798-c265b6b7621f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.291718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c7101-7371-4d92-b798-c265b6b7621f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.303620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c7101-7371-4d92-b798-c265b6b7621f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:26.305121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c7101-7371-4d92-b798-c265b6b7621f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-005c7101-7371-4d92-b798-c265b6b7621f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:26.319043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:26.320163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:26.321077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:26.321077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:26.321077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:26.321077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:26.321077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:26.321077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:26.321077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:26.321077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:26.321077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 387/772] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:26.329189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c7101-7371-4d92-b798-c265b6b7621f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:26.357321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 386/773] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:26.383811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809232a-5daa-4ec8-819b-439de4cc5637 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:26.439956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:26.469727 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c7101-7371-4d92-b798-c265b6b7621f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:26.497371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809232a-5daa-4ec8-819b-439de4cc5637 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:26.517249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9b026-d979-4b41-a039-98e9cd8543f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:26.518053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9b026-d979-4b41-a039-98e9cd8543f4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.518281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9b026-d979-4b41-a039-98e9cd8543f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.518559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9b026-d979-4b41-a039-98e9cd8543f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.519403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9b026-d979-4b41-a039-98e9cd8543f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:26.519713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9b026-d979-4b41-a039-98e9cd8543f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:26.521220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9b026-d979-4b41-a039-98e9cd8543f4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:26.522596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:26.536724 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 388/774] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:26.544687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9b026-d979-4b41-a039-98e9cd8543f4 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:26.553536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec7f91-f0d1-4788-9dc0-dff4b0c0d8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:26.554570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec7f91-f0d1-4788-9dc0-dff4b0c0d8af None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.555300 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec7f91-f0d1-4788-9dc0-dff4b0c0d8af None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.555777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cec7f91-f0d1-4788-9dc0-dff4b0c0d8af None admin] PATH_INFO: `/v3/services/network` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.557127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec7f91-f0d1-4788-9dc0-dff4b0c0d8af None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:26.564256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cec7f91-f0d1-4788-9dc0-dff4b0c0d8af None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-4cec7f91-f0d1-4788-9dc0-dff4b0c0d8af None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:26.578423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:26.579458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 11:51:26.580285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:26.583427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 387/775] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:26 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:26.593538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec7f91-f0d1-4788-9dc0-dff4b0c0d8af None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:26.671526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9b026-d979-4b41-a039-98e9cd8543f4 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:26.739319 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b29206-0f41-499e-a25d-3cd054ffd094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:26.739939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b29206-0f41-499e-a25d-3cd054ffd094 None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.740220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b29206-0f41-499e-a25d-3cd054ffd094 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.740635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b29206-0f41-499e-a25d-3cd054ffd094 None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/db3ff1b21c784e8cb5bffcb79f230d08/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.743943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b29206-0f41-499e-a25d-3cd054ffd094 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=db3ff1b21c784e8cb5bffcb79f230d08, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:26.745680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b29206-0f41-499e-a25d-3cd054ffd094 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:26.761146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:26.768692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cec7f91-f0d1-4788-9dc0-dff4b0c0d8af None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:26.774349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 389/776] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:51:26 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/db3ff1b21c784e8cb5bffcb79f230d08/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:51:26.810073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b967780-693f-456d-8753-73c848e7ca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:26.810728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b967780-693f-456d-8753-73c848e7ca17 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.810996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b967780-693f-456d-8753-73c848e7ca17 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.811261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b967780-693f-456d-8753-73c848e7ca17 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.811947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b967780-693f-456d-8753-73c848e7ca17 None admin] RBAC: Adding query filter params (name=network) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:26.812250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b967780-693f-456d-8753-73c848e7ca17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:26.813702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b967780-693f-456d-8753-73c848e7ca17 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:26.819573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 388/777] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:26.825047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b29206-0f41-499e-a25d-3cd054ffd094 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:26.861507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb779070-a0a6-4bae-acba-1f01c4e0c496 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.861785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb779070-a0a6-4bae-acba-1f01c4e0c496 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.862062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb779070-a0a6-4bae-acba-1f01c4e0c496 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.863881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 389/778] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:26.869607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4e865-72ad-44ac-b933-be838169f01a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.869907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4e865-72ad-44ac-b933-be838169f01a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.870186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4e865-72ad-44ac-b933-be838169f01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.925530 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3ff4e865-72ad-44ac-b933-be838169f01a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:26.926777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3ff4e865-72ad-44ac-b933-be838169f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:26.956502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b29206-0f41-499e-a25d-3cd054ffd094 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:26.994546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d061c45-c44e-4b3e-8110-76d114e2e247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:26.995108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d061c45-c44e-4b3e-8110-76d114e2e247 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:26.995392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d061c45-c44e-4b3e-8110-76d114e2e247 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:26.995667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d061c45-c44e-4b3e-8110-76d114e2e247 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:26.996399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d061c45-c44e-4b3e-8110-76d114e2e247 None admin] RBAC: Adding query filter params (type=network) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:26.996712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d061c45-c44e-4b3e-8110-76d114e2e247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:26.998252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d061c45-c44e-4b3e-8110-76d114e2e247 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:27.004462 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 390/779] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:27.010652 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d061c45-c44e-4b3e-8110-76d114e2e247 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:27.044605 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:27.075379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 390/780] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:27.100160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:27.100964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4e865-72ad-44ac-b933-be838169f01a None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:27.144886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d061c45-c44e-4b3e-8110-76d114e2e247 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:27.185904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da440629-dfa7-4b13-8454-0a1a41f07731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:27.186554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da440629-dfa7-4b13-8454-0a1a41f07731 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:27.186961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da440629-dfa7-4b13-8454-0a1a41f07731 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:27.187237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da440629-dfa7-4b13-8454-0a1a41f07731 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:27.188010 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da440629-dfa7-4b13-8454-0a1a41f07731 None admin] RBAC: Adding query filter params (interface=public, service_id=f7337457188c4d178e8217ab1ebdfcb5, region_id=RegionOne) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:27.188340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da440629-dfa7-4b13-8454-0a1a41f07731 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:27.189736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da440629-dfa7-4b13-8454-0a1a41f07731 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:27.197927 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 391/781] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Aug 1 11:51:27 2022] GET /identity/v3/endpoints?service_id=f7337457188c4d178e8217ab1ebdfcb5&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:27.203233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da440629-dfa7-4b13-8454-0a1a41f07731 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:27.216397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4e865-72ad-44ac-b933-be838169f01a None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:27.255388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1e4df-4700-4c2b-b319-31e26747e0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:27.256543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1e4df-4700-4c2b-b319-31e26747e0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:27.257022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1e4df-4700-4c2b-b319-31e26747e0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:27.257501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1e4df-4700-4c2b-b319-31e26747e0a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:27.268873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1e4df-4700-4c2b-b319-31e26747e0a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-29a1e4df-4700-4c2b-b319-31e26747e0a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:27.274373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:27.275325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:27.276153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:27.278896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 391/782] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:27.284897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1e4df-4700-4c2b-b319-31e26747e0a8 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:27.328424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:27.387460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da440629-dfa7-4b13-8454-0a1a41f07731 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:27.397166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1e4df-4700-4c2b-b319-31e26747e0a8 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:27.435820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcfaf9-e5c2-46b1-b119-b3e3b052afca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:27.436880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcfaf9-e5c2-46b1-b119-b3e3b052afca None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:27.437335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcfaf9-e5c2-46b1-b119-b3e3b052afca None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:27.437777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcfaf9-e5c2-46b1-b119-b3e3b052afca None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:27.438792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcfaf9-e5c2-46b1-b119-b3e3b052afca None admin] RBAC: Adding query filter params (name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:27.439274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcfaf9-e5c2-46b1-b119-b3e3b052afca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:27.441006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcfaf9-e5c2-46b1-b119-b3e3b052afca None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:27.450691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 392/783] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:27.455631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:27.457666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcfaf9-e5c2-46b1-b119-b3e3b052afca None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:27.470076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6536bf-f9a7-44cf-9aa0-7ec707ad47c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:27.470706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6536bf-f9a7-44cf-9aa0-7ec707ad47c0 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:27.470982 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6536bf-f9a7-44cf-9aa0-7ec707ad47c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:27.471251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6536bf-f9a7-44cf-9aa0-7ec707ad47c0 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:27.471952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6536bf-f9a7-44cf-9aa0-7ec707ad47c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:27.473468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6536bf-f9a7-44cf-9aa0-7ec707ad47c0 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:27.479323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 392/784] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Aug 1 11:51:27 2022] GET /identity/v3/services => generated 772 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:27.480623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf255b1-ac7e-4b42-8e27-8c6efa498ca6 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:27.480907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf255b1-ac7e-4b42-8e27-8c6efa498ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:27.481182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf255b1-ac7e-4b42-8e27-8c6efa498ca6 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:27.482469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 393/785] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:27.490769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2ab9e-73ae-4a6b-a5ae-4231b63e986c None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:27.491075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2ab9e-73ae-4a6b-a5ae-4231b63e986c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:27.491309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2ab9e-73ae-4a6b-a5ae-4231b63e986c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:27.513096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-f8f2ab9e-73ae-4a6b-a5ae-4231b63e986c None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:27.514277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f8f2ab9e-73ae-4a6b-a5ae-4231b63e986c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:27.598175 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 394/786] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:27.625668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2ab9e-73ae-4a6b-a5ae-4231b63e986c None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:27.645601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:27.652117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcfaf9-e5c2-46b1-b119-b3e3b052afca None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:27.726013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fe4eb-fd33-42ef-ab78-662865e03d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:27.726776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fe4eb-fd33-42ef-ab78-662865e03d7f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:27.727160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fe4eb-fd33-42ef-ab78-662865e03d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:27.727441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fe4eb-fd33-42ef-ab78-662865e03d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:27.741630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fe4eb-fd33-42ef-ab78-662865e03d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:27.744029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fe4eb-fd33-42ef-ab78-662865e03d7f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-880fe4eb-fd33-42ef-ab78-662865e03d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:27.764882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:27.765828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:27.766728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:27.766728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:27.766728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:27.766728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:27.771368 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 393/787] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:27.773942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fe4eb-fd33-42ef-ab78-662865e03d7f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:27.781712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2ab9e-73ae-4a6b-a5ae-4231b63e986c None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:27.805609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:27.820524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1093613-8bfa-4f97-8935-a8fbb5c40775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:27.821352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1093613-8bfa-4f97-8935-a8fbb5c40775 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:27.821725 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1093613-8bfa-4f97-8935-a8fbb5c40775 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:27.822000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1093613-8bfa-4f97-8935-a8fbb5c40775 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:27.835459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1093613-8bfa-4f97-8935-a8fbb5c40775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-e1093613-8bfa-4f97-8935-a8fbb5c40775 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:27.840463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:27.841403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:27.842234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 395/788] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:27.847597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1093613-8bfa-4f97-8935-a8fbb5c40775 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:27.920973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fe4eb-fd33-42ef-ab78-662865e03d7f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:27.967954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c723bb-ec82-47c1-b58b-e3d2a83d7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:27.969463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c723bb-ec82-47c1-b58b-e3d2a83d7c81 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:27.969853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c723bb-ec82-47c1-b58b-e3d2a83d7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:27.970234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c723bb-ec82-47c1-b58b-e3d2a83d7c81 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:27.971082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c723bb-ec82-47c1-b58b-e3d2a83d7c81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:27.971523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c723bb-ec82-47c1-b58b-e3d2a83d7c81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:27.973150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c723bb-ec82-47c1-b58b-e3d2a83d7c81 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:27.989004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 394/789] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:27.995425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c723bb-ec82-47c1-b58b-e3d2a83d7c81 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:28.022798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:28.040897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1093613-8bfa-4f97-8935-a8fbb5c40775 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:28.077307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258677a-6585-420d-b09f-2ee5a940c4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:28.078140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258677a-6585-420d-b09f-2ee5a940c4dd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:28.078523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258677a-6585-420d-b09f-2ee5a940c4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:28.078891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258677a-6585-420d-b09f-2ee5a940c4dd None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:28.079807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258677a-6585-420d-b09f-2ee5a940c4dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:28.080174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258677a-6585-420d-b09f-2ee5a940c4dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:28.081806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258677a-6585-420d-b09f-2ee5a940c4dd None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:28.091691 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 396/790] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:28.100733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258677a-6585-420d-b09f-2ee5a940c4dd None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:28.172950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c723bb-ec82-47c1-b58b-e3d2a83d7c81 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:28.183284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:28.233506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6dc0-e377-4249-b084-5a0e352f4e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:28.234779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6dc0-e377-4249-b084-5a0e352f4e95 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:28.234893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6dc0-e377-4249-b084-5a0e352f4e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:28.235156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae6dc0-e377-4249-b084-5a0e352f4e95 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:28.245916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6dc0-e377-4249-b084-5a0e352f4e95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:28.247399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae6dc0-e377-4249-b084-5a0e352f4e95 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-ddae6dc0-e377-4249-b084-5a0e352f4e95 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:28.268614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:28.269663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:28.270600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 395/791] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Aug 1 11:51:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:28.277024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6dc0-e377-4249-b084-5a0e352f4e95 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:28.280599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258677a-6585-420d-b09f-2ee5a940c4dd None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:28.357481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34291a6c-1c88-4d03-8760-b2b0b3bdf8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:28.358138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34291a6c-1c88-4d03-8760-b2b0b3bdf8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:28.358419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34291a6c-1c88-4d03-8760-b2b0b3bdf8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:28.358695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34291a6c-1c88-4d03-8760-b2b0b3bdf8a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:28.384254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34291a6c-1c88-4d03-8760-b2b0b3bdf8a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:28.385729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34291a6c-1c88-4d03-8760-b2b0b3bdf8a9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:28.398482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-34291a6c-1c88-4d03-8760-b2b0b3bdf8a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:28.421790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:28.424214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:28.426417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:28.426417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:28.426417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:28.426417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:28.426417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 397/792] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:28.433282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34291a6c-1c88-4d03-8760-b2b0b3bdf8a9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:28.435849 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae6dc0-e377-4249-b084-5a0e352f4e95 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:28.481945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5db74-64a4-464e-af4c-cf8b706240c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:28.483639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5db74-64a4-464e-af4c-cf8b706240c4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:28.483922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5db74-64a4-464e-af4c-cf8b706240c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:28.484511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d5db74-64a4-464e-af4c-cf8b706240c4 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:28.485350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5db74-64a4-464e-af4c-cf8b706240c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:28.485657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5db74-64a4-464e-af4c-cf8b706240c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:28.487111 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d5db74-64a4-464e-af4c-cf8b706240c4 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:28.538191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 396/793] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Aug 1 11:51:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:28.541978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5db74-64a4-464e-af4c-cf8b706240c4 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:28.620009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:28.629080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34291a6c-1c88-4d03-8760-b2b0b3bdf8a9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:28.668946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d5db74-64a4-464e-af4c-cf8b706240c4 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:28.675240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e1113-fd71-4ce3-8891-e4970e22d962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:28.675870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e1113-fd71-4ce3-8891-e4970e22d962 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:28.676159 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e1113-fd71-4ce3-8891-e4970e22d962 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:28.676458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e1113-fd71-4ce3-8891-e4970e22d962 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:28.677167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e1113-fd71-4ce3-8891-e4970e22d962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:28.677471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e1113-fd71-4ce3-8891-e4970e22d962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:28.678862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e1113-fd71-4ce3-8891-e4970e22d962 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:28.697891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 398/794] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:28.699651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786237f-e340-4026-b841-90396d4f63eb None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:28.700075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786237f-e340-4026-b841-90396d4f63eb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:28.700565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786237f-e340-4026-b841-90396d4f63eb None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:28.708014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 399/795] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:28.708014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786237f-e340-4026-b841-90396d4f63eb None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:28.716931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:28.723581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24710dd4-4883-4740-b09f-428bce818832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:28.724236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24710dd4-4883-4740-b09f-428bce818832 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:28.724547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24710dd4-4883-4740-b09f-428bce818832 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:28.724819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24710dd4-4883-4740-b09f-428bce818832 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:28.742277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24710dd4-4883-4740-b09f-428bce818832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:28.744064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24710dd4-4883-4740-b09f-428bce818832 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-24710dd4-4883-4740-b09f-428bce818832 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:28.761498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:28.762649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:28.763784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:28.763784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:28.763784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:28.763784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:28.763784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 397/796] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:28.764544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54b14b-60d2-45a6-ab0f-a116852f4829 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:28.764825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54b14b-60d2-45a6-ab0f-a116852f4829 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:28.765138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a54b14b-60d2-45a6-ab0f-a116852f4829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:28.788559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-9a54b14b-60d2-45a6-ab0f-a116852f4829 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:28.790153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9a54b14b-60d2-45a6-ab0f-a116852f4829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:28.871795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786237f-e340-4026-b841-90396d4f63eb None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:28.876732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 398/797] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:28.877823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54b14b-60d2-45a6-ab0f-a116852f4829 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:28.925873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7ecad-f80e-4ab5-be86-5c6b9a2526e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:28.927334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7ecad-f80e-4ab5-be86-5c6b9a2526e2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:28.928130 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7ecad-f80e-4ab5-be86-5c6b9a2526e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:28.928638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c7ecad-f80e-4ab5-be86-5c6b9a2526e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:28.940803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7ecad-f80e-4ab5-be86-5c6b9a2526e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:28.942330 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c7ecad-f80e-4ab5-be86-5c6b9a2526e2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-50c7ecad-f80e-4ab5-be86-5c6b9a2526e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:28.963672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:28.965952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 400/798] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:28.968158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7ecad-f80e-4ab5-be86-5c6b9a2526e2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:28.990377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:29.037890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a54b14b-60d2-45a6-ab0f-a116852f4829 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:29.054866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:29.082763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e96174-d8bd-4770-8250-237ddc6ff53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:29.083469 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96174-d8bd-4770-8250-237ddc6ff53c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.084078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96174-d8bd-4770-8250-237ddc6ff53c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.084078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e96174-d8bd-4770-8250-237ddc6ff53c None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.085181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e96174-d8bd-4770-8250-237ddc6ff53c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:29.085591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e96174-d8bd-4770-8250-237ddc6ff53c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:29.088199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e96174-d8bd-4770-8250-237ddc6ff53c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:29.091344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c7ecad-f80e-4ab5-be86-5c6b9a2526e2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:29.102185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 399/799] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:29.103156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e96174-d8bd-4770-8250-237ddc6ff53c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:29.138981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f1a83-93f1-4377-8e07-0c55c72a4782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:29.140535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1a83-93f1-4377-8e07-0c55c72a4782 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.141250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1a83-93f1-4377-8e07-0c55c72a4782 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.141911 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f1a83-93f1-4377-8e07-0c55c72a4782 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.143234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f1a83-93f1-4377-8e07-0c55c72a4782 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:29.144858 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7f1a83-93f1-4377-8e07-0c55c72a4782 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-4d7f1a83-93f1-4377-8e07-0c55c72a4782 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:29.160582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:29.161693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 01 11:51:29.162917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:29.165455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 401/800] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Aug 1 11:51:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:29.166444 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f1a83-93f1-4377-8e07-0c55c72a4782 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:29.222202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e96174-d8bd-4770-8250-237ddc6ff53c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:29.253612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:29.274650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210855-8e4a-49a2-b416-0002f414d5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:29.275301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210855-8e4a-49a2-b416-0002f414d5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.275611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210855-8e4a-49a2-b416-0002f414d5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.276086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210855-8e4a-49a2-b416-0002f414d5e5 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.277533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210855-8e4a-49a2-b416-0002f414d5e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:29.277713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210855-8e4a-49a2-b416-0002f414d5e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:29.279518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210855-8e4a-49a2-b416-0002f414d5e5 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:29.280780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7f1a83-93f1-4377-8e07-0c55c72a4782 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:29.310985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 400/801] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Aug 1 11:51:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:29.310985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210855-8e4a-49a2-b416-0002f414d5e5 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:29.325460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90fad5a-7624-4b71-b5a5-7daf9fda67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:29.326616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fad5a-7624-4b71-b5a5-7daf9fda67b8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.327081 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fad5a-7624-4b71-b5a5-7daf9fda67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.327503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fad5a-7624-4b71-b5a5-7daf9fda67b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.336056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90fad5a-7624-4b71-b5a5-7daf9fda67b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:29.337887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90fad5a-7624-4b71-b5a5-7daf9fda67b8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-e90fad5a-7624-4b71-b5a5-7daf9fda67b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:29.350938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:29.352075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:29.353087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:29.353087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:29.353087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:29.353087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:29.353087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:29.353087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:29.353087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:29.353087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:29.355254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:29.355602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 402/802] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:29.357225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90fad5a-7624-4b71-b5a5-7daf9fda67b8 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:29.446512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210855-8e4a-49a2-b416-0002f414d5e5 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:29.492168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd831c-a8f4-4022-b188-179a2732c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:29.492911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd831c-a8f4-4022-b188-179a2732c35b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.493096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd831c-a8f4-4022-b188-179a2732c35b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.493366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd831c-a8f4-4022-b188-179a2732c35b None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.494055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd831c-a8f4-4022-b188-179a2732c35b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:29.494359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd831c-a8f4-4022-b188-179a2732c35b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:29.495751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd831c-a8f4-4022-b188-179a2732c35b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:29.498670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90fad5a-7624-4b71-b5a5-7daf9fda67b8 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:29.501335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 401/803] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Aug 1 11:51:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:29.506669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd831c-a8f4-4022-b188-179a2732c35b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:29.545582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1f8f6-ea99-4b48-880d-ceedce815741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:29.546250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1f8f6-ea99-4b48-880d-ceedce815741 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.546529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1f8f6-ea99-4b48-880d-ceedce815741 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.546802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1f8f6-ea99-4b48-880d-ceedce815741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.555906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:29.560346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1f8f6-ea99-4b48-880d-ceedce815741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:29.562540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1f8f6-ea99-4b48-880d-ceedce815741 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-25a1f8f6-ea99-4b48-880d-ceedce815741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:29.583518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:29.584791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:29.585924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:29.585924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:29.585924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:29.585924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:29.585924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 403/804] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:29.586391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396408-0e12-4623-8905-b735adc57d04 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.587353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396408-0e12-4623-8905-b735adc57d04 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.587353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396408-0e12-4623-8905-b735adc57d04 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.589692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 404/805] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:29.589993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396408-0e12-4623-8905-b735adc57d04 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:29.629407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:29.638276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd831c-a8f4-4022-b188-179a2732c35b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:29.683063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79dd93-6bee-4110-b0b0-eed6d3714f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:29.684191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79dd93-6bee-4110-b0b0-eed6d3714f6b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.684508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79dd93-6bee-4110-b0b0-eed6d3714f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.684762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79dd93-6bee-4110-b0b0-eed6d3714f6b None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.685988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79dd93-6bee-4110-b0b0-eed6d3714f6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:29.686306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79dd93-6bee-4110-b0b0-eed6d3714f6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:29.688407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79dd93-6bee-4110-b0b0-eed6d3714f6b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:29.702851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 402/806] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:29.703707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79dd93-6bee-4110-b0b0-eed6d3714f6b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:29.717764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396408-0e12-4623-8905-b735adc57d04 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:29.753993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706b93d-7cee-474e-9b3e-6f2079369614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:29.754648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706b93d-7cee-474e-9b3e-6f2079369614 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.754968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706b93d-7cee-474e-9b3e-6f2079369614 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.755244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706b93d-7cee-474e-9b3e-6f2079369614 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.755930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706b93d-7cee-474e-9b3e-6f2079369614 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:29.756263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706b93d-7cee-474e-9b3e-6f2079369614 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:29.757808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0706b93d-7cee-474e-9b3e-6f2079369614 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:29.767795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 405/807] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Aug 1 11:51:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:29.767795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164af57-334a-4691-a9a2-b7b396ff1a75 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.767795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164af57-334a-4691-a9a2-b7b396ff1a75 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.767795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164af57-334a-4691-a9a2-b7b396ff1a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.787146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-5164af57-334a-4691-a9a2-b7b396ff1a75 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:29.788521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-5164af57-334a-4691-a9a2-b7b396ff1a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:29.825917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79dd93-6bee-4110-b0b0-eed6d3714f6b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:29.849711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:29.871932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6e63e-b770-4869-b9de-f8dfe5c22284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:29.872610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6e63e-b770-4869-b9de-f8dfe5c22284 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:29.872886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6e63e-b770-4869-b9de-f8dfe5c22284 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:29.873158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee6e63e-b770-4869-b9de-f8dfe5c22284 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:29.873352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 406/808] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:29.873924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6e63e-b770-4869-b9de-f8dfe5c22284 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:29.874152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164af57-334a-4691-a9a2-b7b396ff1a75 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:29.874328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6e63e-b770-4869-b9de-f8dfe5c22284 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:29.876027 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee6e63e-b770-4869-b9de-f8dfe5c22284 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:29.896073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 403/809] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:29.896815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6e63e-b770-4869-b9de-f8dfe5c22284 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:29.913670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:29.990903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164af57-334a-4691-a9a2-b7b396ff1a75 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:30.008043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee6e63e-b770-4869-b9de-f8dfe5c22284 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:30.030840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94f565-a9e3-416a-b7a0-7032d56f501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:30.031516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94f565-a9e3-416a-b7a0-7032d56f501f None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.031797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94f565-a9e3-416a-b7a0-7032d56f501f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.032041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94f565-a9e3-416a-b7a0-7032d56f501f None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/bf76a8ec03e444e7b42e2997b8f73808/roles/a55004b069fa4cd99327dd3d4f55efcf` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.050086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94f565-a9e3-416a-b7a0-7032d56f501f None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=bf76a8ec03e444e7b42e2997b8f73808, role_id=a55004b069fa4cd99327dd3d4f55efcf)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:30.051578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94f565-a9e3-416a-b7a0-7032d56f501f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:30.054555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d1e8c-d9f2-4671-8225-326875636ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:30.055177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d1e8c-d9f2-4671-8225-326875636ae2 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.055465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d1e8c-d9f2-4671-8225-326875636ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.055745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d1e8c-d9f2-4671-8225-326875636ae2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.056495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d1e8c-d9f2-4671-8225-326875636ae2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:30.057879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2d1e8c-d9f2-4671-8225-326875636ae2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:30.066389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 407/810] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:51:29 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/bf76a8ec03e444e7b42e2997b8f73808/roles/a55004b069fa4cd99327dd3d4f55efcf => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:51:30.067642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ccefa-12a1-417f-9e04-d85b4d2a2633 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.067924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ccefa-12a1-417f-9e04-d85b4d2a2633 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.068197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ccefa-12a1-417f-9e04-d85b4d2a2633 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.069896 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 408/811] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:30.070261 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ccefa-12a1-417f-9e04-d85b4d2a2633 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:30.086044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 404/812] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 11:51:29 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:30.086722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d1e8c-d9f2-4671-8225-326875636ae2 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:30.129834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:30.218491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ccefa-12a1-417f-9e04-d85b4d2a2633 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:30.223293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:30.270716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a54b87-a8e0-478c-91d8-e17fb2412a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:30.271457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a54b87-a8e0-478c-91d8-e17fb2412a86 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.271819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a54b87-a8e0-478c-91d8-e17fb2412a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.272009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a54b87-a8e0-478c-91d8-e17fb2412a86 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.281250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d1e8c-d9f2-4671-8225-326875636ae2 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:30.288994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a54b87-a8e0-478c-91d8-e17fb2412a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-e9a54b87-a8e0-478c-91d8-e17fb2412a86 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:30.293803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:30.295611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 409/813] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:30.297048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42998cf2-470b-46fe-a314-6d02c607bccb None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.297817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42998cf2-470b-46fe-a314-6d02c607bccb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.297817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42998cf2-470b-46fe-a314-6d02c607bccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.322248 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-42998cf2-470b-46fe-a314-6d02c607bccb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:30.323481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-42998cf2-470b-46fe-a314-6d02c607bccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:30.329701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abe6cd-8934-4553-a77d-4f3835baf96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:30.330523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abe6cd-8934-4553-a77d-4f3835baf96b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.331070 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abe6cd-8934-4553-a77d-4f3835baf96b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.331617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abe6cd-8934-4553-a77d-4f3835baf96b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.343839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abe6cd-8934-4553-a77d-4f3835baf96b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:30.346028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abe6cd-8934-4553-a77d-4f3835baf96b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-36abe6cd-8934-4553-a77d-4f3835baf96b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:30.363342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:30.364463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:30.367056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:30.367056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:30.367056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:30.367056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:30.367056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:30.367056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:30.367056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:30.367056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:30.367056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 405/814] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:30.367956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abe6cd-8934-4553-a77d-4f3835baf96b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:30.425977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 410/815] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:30.426864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42998cf2-470b-46fe-a314-6d02c607bccb None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:30.451351 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:30.477969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abe6cd-8934-4553-a77d-4f3835baf96b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:30.508509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:30.515240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548c806-c1a4-4ac9-a164-a972e77d4e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:30.515884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548c806-c1a4-4ac9-a164-a972e77d4e58 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.516165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548c806-c1a4-4ac9-a164-a972e77d4e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.516476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3548c806-c1a4-4ac9-a164-a972e77d4e58 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.517205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548c806-c1a4-4ac9-a164-a972e77d4e58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:30.517513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548c806-c1a4-4ac9-a164-a972e77d4e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:30.518948 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3548c806-c1a4-4ac9-a164-a972e77d4e58 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:30.530058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 406/816] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:30.535863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548c806-c1a4-4ac9-a164-a972e77d4e58 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:30.570813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42998cf2-470b-46fe-a314-6d02c607bccb None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:30.625040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710f120-e97c-4174-b8e3-1daab8d1561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:30.626213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710f120-e97c-4174-b8e3-1daab8d1561c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.628612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710f120-e97c-4174-b8e3-1daab8d1561c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.629104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710f120-e97c-4174-b8e3-1daab8d1561c None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.632629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710f120-e97c-4174-b8e3-1daab8d1561c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:30.633178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710f120-e97c-4174-b8e3-1daab8d1561c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:30.635084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710f120-e97c-4174-b8e3-1daab8d1561c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:30.659817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3548c806-c1a4-4ac9-a164-a972e77d4e58 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:30.661421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 411/817] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:30.662561 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710f120-e97c-4174-b8e3-1daab8d1561c None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:30.699952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bf0df-7764-47cf-b414-63fe10804f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:30.701167 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bf0df-7764-47cf-b414-63fe10804f0e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.701624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bf0df-7764-47cf-b414-63fe10804f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.702209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bf0df-7764-47cf-b414-63fe10804f0e None admin] PATH_INFO: `/v3/services/network` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.703070 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bf0df-7764-47cf-b414-63fe10804f0e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:30.704801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bf0df-7764-47cf-b414-63fe10804f0e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-989bf0df-7764-47cf-b414-63fe10804f0e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:30.715537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:30.716616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 01 11:51:30.717459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:30.719940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 407/818] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:30 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:30.721017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bf0df-7764-47cf-b414-63fe10804f0e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:30.782671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:30.799580 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:30.816874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710f120-e97c-4174-b8e3-1daab8d1561c None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:30.866169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bf0df-7764-47cf-b414-63fe10804f0e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:30.868384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bd8e6-7e69-43a0-926c-0d2ed7ed0a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:30.869243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bd8e6-7e69-43a0-926c-0d2ed7ed0a58 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.869531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bd8e6-7e69-43a0-926c-0d2ed7ed0a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.869804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bd8e6-7e69-43a0-926c-0d2ed7ed0a58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.882501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03bd8e6-7e69-43a0-926c-0d2ed7ed0a58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:30.884219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03bd8e6-7e69-43a0-926c-0d2ed7ed0a58 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-d03bd8e6-7e69-43a0-926c-0d2ed7ed0a58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:30.902346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:30.904051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:30.905326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:30.905326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:30.905326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:30.905326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:30.908443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 412/819] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:30.911384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bd8e6-7e69-43a0-926c-0d2ed7ed0a58 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:30.929134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596ecef-4654-4395-822c-8a8d2c85d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:30.929839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ecef-4654-4395-822c-8a8d2c85d6df None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:30.930125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ecef-4654-4395-822c-8a8d2c85d6df None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:30.930405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ecef-4654-4395-822c-8a8d2c85d6df None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/b9b6a96421e349a88a38590cfce167fb/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:30.933275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596ecef-4654-4395-822c-8a8d2c85d6df None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=b9b6a96421e349a88a38590cfce167fb, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:30.934939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596ecef-4654-4395-822c-8a8d2c85d6df None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:30.960393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 408/820] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:51:30 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/b9b6a96421e349a88a38590cfce167fb/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:51:30.960393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596ecef-4654-4395-822c-8a8d2c85d6df None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:31.100511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bd8e6-7e69-43a0-926c-0d2ed7ed0a58 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:31.127552 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596ecef-4654-4395-822c-8a8d2c85d6df None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:31.148796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:31.157834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5f1c0-5fb1-4fac-958c-7aab39bd6294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:31.158632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5f1c0-5fb1-4fac-958c-7aab39bd6294 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:31.158940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5f1c0-5fb1-4fac-958c-7aab39bd6294 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:31.159214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde5f1c0-5fb1-4fac-958c-7aab39bd6294 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:31.160115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5f1c0-5fb1-4fac-958c-7aab39bd6294 None admin] RBAC: Adding query filter params (name=network) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:31.160450 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5f1c0-5fb1-4fac-958c-7aab39bd6294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:31.163974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde5f1c0-5fb1-4fac-958c-7aab39bd6294 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:31.170316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:31.181674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 413/821] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:31.181949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5f1c0-5fb1-4fac-958c-7aab39bd6294 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:31.202589 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6aad2-797e-4be3-9fdb-f228b43f859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:31.203385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6aad2-797e-4be3-9fdb-f228b43f859d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:31.203795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6aad2-797e-4be3-9fdb-f228b43f859d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:31.204615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6aad2-797e-4be3-9fdb-f228b43f859d None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:31.205595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6aad2-797e-4be3-9fdb-f228b43f859d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:31.206103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6aad2-797e-4be3-9fdb-f228b43f859d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:31.207638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6aad2-797e-4be3-9fdb-f228b43f859d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:31.224947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 409/822] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:31.232325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6aad2-797e-4be3-9fdb-f228b43f859d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:31.330937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde5f1c0-5fb1-4fac-958c-7aab39bd6294 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:31.393978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c53077-155d-4eb8-aaae-b7bfcb44775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:31.395833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c53077-155d-4eb8-aaae-b7bfcb44775c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:31.396119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c53077-155d-4eb8-aaae-b7bfcb44775c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:31.396530 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c53077-155d-4eb8-aaae-b7bfcb44775c None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:31.397115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c53077-155d-4eb8-aaae-b7bfcb44775c None admin] RBAC: Adding query filter params (type=network) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:31.397417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c53077-155d-4eb8-aaae-b7bfcb44775c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:31.399050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c53077-155d-4eb8-aaae-b7bfcb44775c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:31.408694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 414/823] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:31.410844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6aad2-797e-4be3-9fdb-f228b43f859d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:31.415902 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c53077-155d-4eb8-aaae-b7bfcb44775c None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:31.474390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780cd9e-1efa-4d28-8f2c-088bcea51088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:31.476163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780cd9e-1efa-4d28-8f2c-088bcea51088 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:31.476470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780cd9e-1efa-4d28-8f2c-088bcea51088 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:31.476744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780cd9e-1efa-4d28-8f2c-088bcea51088 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:31.491208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780cd9e-1efa-4d28-8f2c-088bcea51088 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:31.494602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780cd9e-1efa-4d28-8f2c-088bcea51088 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-6780cd9e-1efa-4d28-8f2c-088bcea51088 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:31.518955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:31.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:31.521313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 410/824] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Aug 1 11:51:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:31.526007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:31.528308 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780cd9e-1efa-4d28-8f2c-088bcea51088 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:31.574400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:31.625196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c53077-155d-4eb8-aaae-b7bfcb44775c None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:31.675042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780cd9e-1efa-4d28-8f2c-088bcea51088 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:31.684914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb6117-f41b-49e4-97b6-ecec0be7a635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:31.685565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb6117-f41b-49e4-97b6-ecec0be7a635 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:31.685854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb6117-f41b-49e4-97b6-ecec0be7a635 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:31.686130 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb6117-f41b-49e4-97b6-ecec0be7a635 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:31.686863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb6117-f41b-49e4-97b6-ecec0be7a635 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:31.688413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb6117-f41b-49e4-97b6-ecec0be7a635 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:31.716480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54021ae-ae49-454b-b0ff-eab5944ce43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:31.717273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54021ae-ae49-454b-b0ff-eab5944ce43e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:31.717621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54021ae-ae49-454b-b0ff-eab5944ce43e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:31.717959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54021ae-ae49-454b-b0ff-eab5944ce43e None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:31.719447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54021ae-ae49-454b-b0ff-eab5944ce43e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:31.719953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54021ae-ae49-454b-b0ff-eab5944ce43e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:31.721675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54021ae-ae49-454b-b0ff-eab5944ce43e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:31.739739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 415/825] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 11:51:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:31.749071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 411/826] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Aug 1 11:51:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:31.755734 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb6117-f41b-49e4-97b6-ecec0be7a635 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:31.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb6117-f41b-49e4-97b6-ecec0be7a635 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:31.950626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:31.976552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b1b8a-a21a-4d8e-a1ae-7e1a32da0bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:31.977491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b1b8a-a21a-4d8e-a1ae-7e1a32da0bcb None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:31.977909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b1b8a-a21a-4d8e-a1ae-7e1a32da0bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:31.978344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b1b8a-a21a-4d8e-a1ae-7e1a32da0bcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:32.060443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b1b8a-a21a-4d8e-a1ae-7e1a32da0bcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:32.062156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b1b8a-a21a-4d8e-a1ae-7e1a32da0bcb None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:32.176536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 416/827] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:31 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:51:32.181805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54021ae-ae49-454b-b0ff-eab5944ce43e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:32.286049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:32.292668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54021ae-ae49-454b-b0ff-eab5944ce43e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:32.328434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b53959-300b-4083-8a75-8f48130eb0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:32.329214 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b53959-300b-4083-8a75-8f48130eb0ee None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:32.329496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b53959-300b-4083-8a75-8f48130eb0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:32.329769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b53959-300b-4083-8a75-8f48130eb0ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:32.340322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b53959-300b-4083-8a75-8f48130eb0ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:32.341946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b53959-300b-4083-8a75-8f48130eb0ee None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-e6b53959-300b-4083-8a75-8f48130eb0ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:32.356959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:32.357904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:32.359373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:32.359373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:32.359373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:32.359373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:32.359373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 412/828] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:32.360862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247f15b-3916-46a7-8618-0efdc803a28d None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:32.361149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247f15b-3916-46a7-8618-0efdc803a28d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:32.361490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247f15b-3916-46a7-8618-0efdc803a28d None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:32.363278 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 417/829] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:32.366121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b53959-300b-4083-8a75-8f48130eb0ee None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:32.368176 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ca3a3-479a-40a7-9b0d-418163b38a2a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:32.368502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ca3a3-479a-40a7-9b0d-418163b38a2a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:32.368776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ca3a3-479a-40a7-9b0d-418163b38a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:32.388532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2b5ca3a3-479a-40a7-9b0d-418163b38a2a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:32.389742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2b5ca3a3-479a-40a7-9b0d-418163b38a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:32.397929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:32.465607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 418/830] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:32.471393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b53959-300b-4083-8a75-8f48130eb0ee None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:32.491505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ca3a3-479a-40a7-9b0d-418163b38a2a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:32.515858 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46675fce-59e5-405a-bbc4-e81c601660d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:32.516548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46675fce-59e5-405a-bbc4-e81c601660d4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:32.516829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46675fce-59e5-405a-bbc4-e81c601660d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:32.517099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46675fce-59e5-405a-bbc4-e81c601660d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:32.517823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46675fce-59e5-405a-bbc4-e81c601660d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:32.518125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46675fce-59e5-405a-bbc4-e81c601660d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:32.519627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46675fce-59e5-405a-bbc4-e81c601660d4 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:32.533047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 413/831] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:32.541414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46675fce-59e5-405a-bbc4-e81c601660d4 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:32.550971 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:32.609758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ca3a3-479a-40a7-9b0d-418163b38a2a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:32.648562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd0f13a-fbb5-4341-95d1-d21aa0b8fe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:32.649322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0f13a-fbb5-4341-95d1-d21aa0b8fe97 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:32.649772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0f13a-fbb5-4341-95d1-d21aa0b8fe97 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:32.650050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd0f13a-fbb5-4341-95d1-d21aa0b8fe97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:32.660864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd0f13a-fbb5-4341-95d1-d21aa0b8fe97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-5dd0f13a-fbb5-4341-95d1-d21aa0b8fe97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:32.665904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:32.666873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:32.667695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 419/832] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:51:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:32.669028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acd7cb-9564-4d98-b0eb-7ed287e09fd4 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:32.669028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acd7cb-9564-4d98-b0eb-7ed287e09fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:32.669207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acd7cb-9564-4d98-b0eb-7ed287e09fd4 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:32.670483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 420/833] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:32.672994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acd7cb-9564-4d98-b0eb-7ed287e09fd4 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:32.677630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:32.679693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46675fce-59e5-405a-bbc4-e81c601660d4 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:32.717288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccaba5c-a3ad-4a74-908b-0486d56bf1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:32.718175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaba5c-a3ad-4a74-908b-0486d56bf1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:32.718441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaba5c-a3ad-4a74-908b-0486d56bf1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:32.718680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccaba5c-a3ad-4a74-908b-0486d56bf1a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:32.730058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccaba5c-a3ad-4a74-908b-0486d56bf1a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:32.731725 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccaba5c-a3ad-4a74-908b-0486d56bf1a1 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-9ccaba5c-a3ad-4a74-908b-0486d56bf1a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:32.745853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:32.746774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:32.748306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:32.748306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:32.748306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:32.748306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:32.748306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:32.748306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:32.748306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:32.748306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:32.748306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 414/834] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:32.748817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526cbb7-6111-4887-9fc4-045f25611798 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:32.748892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526cbb7-6111-4887-9fc4-045f25611798 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:32.749149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526cbb7-6111-4887-9fc4-045f25611798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:32.770827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c526cbb7-6111-4887-9fc4-045f25611798 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:32.772055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c526cbb7-6111-4887-9fc4-045f25611798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:32.791214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acd7cb-9564-4d98-b0eb-7ed287e09fd4 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:32.833818 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8c787-0b8a-4e56-8cf5-3e2e0bf3dd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:32.834471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c787-0b8a-4e56-8cf5-3e2e0bf3dd0d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:32.834749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c787-0b8a-4e56-8cf5-3e2e0bf3dd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:32.835022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8c787-0b8a-4e56-8cf5-3e2e0bf3dd0d None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:32.835750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c787-0b8a-4e56-8cf5-3e2e0bf3dd0d None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:32.836051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c787-0b8a-4e56-8cf5-3e2e0bf3dd0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:32.837483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8c787-0b8a-4e56-8cf5-3e2e0bf3dd0d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:32.841980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:32.850843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 415/835] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:32.851788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 421/836] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:51:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:32.856139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526cbb7-6111-4887-9fc4-045f25611798 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:32.857178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8c787-0b8a-4e56-8cf5-3e2e0bf3dd0d None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:32.927183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:32.964335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526cbb7-6111-4887-9fc4-045f25611798 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:32.970865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8c787-0b8a-4e56-8cf5-3e2e0bf3dd0d None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:33.002414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488cafb-6ae3-462f-a32c-5944112d7ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.003078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488cafb-6ae3-462f-a32c-5944112d7ccf None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.003360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488cafb-6ae3-462f-a32c-5944112d7ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.003635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488cafb-6ae3-462f-a32c-5944112d7ccf None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.004427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488cafb-6ae3-462f-a32c-5944112d7ccf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:33.004737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488cafb-6ae3-462f-a32c-5944112d7ccf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:33.005807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1281dd-1ff3-4ea6-8a95-55a1fe4acfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.006182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488cafb-6ae3-462f-a32c-5944112d7ccf None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:33.006455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1281dd-1ff3-4ea6-8a95-55a1fe4acfec None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.006732 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1281dd-1ff3-4ea6-8a95-55a1fe4acfec None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.007004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1281dd-1ff3-4ea6-8a95-55a1fe4acfec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.018152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1281dd-1ff3-4ea6-8a95-55a1fe4acfec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:33.019775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1281dd-1ff3-4ea6-8a95-55a1fe4acfec None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:33.023452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 416/837] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:33.024188 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488cafb-6ae3-462f-a32c-5944112d7ccf None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-bb1281dd-1ff3-4ea6-8a95-55a1fe4acfec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:33.035118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:33.036089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:33.037018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:33.037018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:33.037018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:33.037018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:33.037018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 422/838] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:33.037300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1281dd-1ff3-4ea6-8a95-55a1fe4acfec None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:33.099386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:33.148204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488cafb-6ae3-462f-a32c-5944112d7ccf None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:33.152021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1281dd-1ff3-4ea6-8a95-55a1fe4acfec None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:33.170401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:33.183299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dccd9f-7b7f-4e2f-8b50-5b3eb08a79e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.183957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dccd9f-7b7f-4e2f-8b50-5b3eb08a79e7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.184238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dccd9f-7b7f-4e2f-8b50-5b3eb08a79e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.184538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dccd9f-7b7f-4e2f-8b50-5b3eb08a79e7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.185171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dccd9f-7b7f-4e2f-8b50-5b3eb08a79e7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:33.186550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dccd9f-7b7f-4e2f-8b50-5b3eb08a79e7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:33.191421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21c05e-44af-45a3-b090-fc0e1995f009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.192047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21c05e-44af-45a3-b090-fc0e1995f009 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.192352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21c05e-44af-45a3-b090-fc0e1995f009 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.192627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21c05e-44af-45a3-b090-fc0e1995f009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.193426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21c05e-44af-45a3-b090-fc0e1995f009 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=db3ff1b21c784e8cb5bffcb79f230d08) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:33.193730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21c05e-44af-45a3-b090-fc0e1995f009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:33.195224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b21c05e-44af-45a3-b090-fc0e1995f009 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-05dccd9f-7b7f-4e2f-8b50-5b3eb08a79e7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:33.195748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:33.196881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:33.202946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 417/839] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:51:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:33.203934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dccd9f-7b7f-4e2f-8b50-5b3eb08a79e7 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:33.220669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 423/840] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:33 2022] GET /identity/v3/role_assignments?user.id=db3ff1b21c784e8cb5bffcb79f230d08&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:33.220669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21c05e-44af-45a3-b090-fc0e1995f009 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:33.324811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dccd9f-7b7f-4e2f-8b50-5b3eb08a79e7 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:33.333570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b21c05e-44af-45a3-b090-fc0e1995f009 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:33.347553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:33.365934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84b5a9-a939-45c2-83dc-d01bd0bce1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.366583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84b5a9-a939-45c2-83dc-d01bd0bce1cb None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.366867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84b5a9-a939-45c2-83dc-d01bd0bce1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.367145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84b5a9-a939-45c2-83dc-d01bd0bce1cb None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.367869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84b5a9-a939-45c2-83dc-d01bd0bce1cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:33.368179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84b5a9-a939-45c2-83dc-d01bd0bce1cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:33.369646 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84b5a9-a939-45c2-83dc-d01bd0bce1cb None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:33.378184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70b30-b57b-4104-9141-81906cde63d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.378951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70b30-b57b-4104-9141-81906cde63d6 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.379305 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70b30-b57b-4104-9141-81906cde63d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.379650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70b30-b57b-4104-9141-81906cde63d6 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.380475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e70b30-b57b-4104-9141-81906cde63d6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:33.380848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e70b30-b57b-4104-9141-81906cde63d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:33.382341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e70b30-b57b-4104-9141-81906cde63d6 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:33.383150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 418/841] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:33.388518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84b5a9-a939-45c2-83dc-d01bd0bce1cb None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:33.393623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 424/842] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:51:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:33.399225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70b30-b57b-4104-9141-81906cde63d6 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:33.457581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:33.504255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84b5a9-a939-45c2-83dc-d01bd0bce1cb None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:33.521435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70b30-b57b-4104-9141-81906cde63d6 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:33.538960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734f5e1-eab9-4e1b-af58-7f7b84cf7317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.539615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734f5e1-eab9-4e1b-af58-7f7b84cf7317 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.539893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734f5e1-eab9-4e1b-af58-7f7b84cf7317 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.540167 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734f5e1-eab9-4e1b-af58-7f7b84cf7317 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.547803 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734f5e1-eab9-4e1b-af58-7f7b84cf7317 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:33.549290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6734f5e1-eab9-4e1b-af58-7f7b84cf7317 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:33.556378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a51b3-2ea3-4948-b69b-05ebe5bef36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.557014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a51b3-2ea3-4948-b69b-05ebe5bef36a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.557291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a51b3-2ea3-4948-b69b-05ebe5bef36a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.557562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a51b3-2ea3-4948-b69b-05ebe5bef36a None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.558251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a51b3-2ea3-4948-b69b-05ebe5bef36a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:33.558559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a51b3-2ea3-4948-b69b-05ebe5bef36a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:33.559933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a51b3-2ea3-4948-b69b-05ebe5bef36a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-6734f5e1-eab9-4e1b-af58-7f7b84cf7317 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:33.561099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:33.562197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:33.563327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:33.563327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:33.563327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:33.563327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:33.563327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:33.563327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:33.563327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:33.563327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:33.563327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:33.563327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:33.572322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 425/843] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:51:33 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:33.572322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d364-0dbf-4e68-8eda-c12ab6fa41b7 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.572612 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 419/844] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Aug 1 11:51:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:33.572700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d364-0dbf-4e68-8eda-c12ab6fa41b7 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.572700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d364-0dbf-4e68-8eda-c12ab6fa41b7 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.572874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734f5e1-eab9-4e1b-af58-7f7b84cf7317 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:33.572959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 426/845] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:33.574757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134e7cb-814a-46d2-996e-858b7c03405a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.575163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134e7cb-814a-46d2-996e-858b7c03405a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.575439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134e7cb-814a-46d2-996e-858b7c03405a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.596442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4134e7cb-814a-46d2-996e-858b7c03405a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:33.597921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4134e7cb-814a-46d2-996e-858b7c03405a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:33.621875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:33.690085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734f5e1-eab9-4e1b-af58-7f7b84cf7317 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:33.708317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 427/846] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:33 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:33.723819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134e7cb-814a-46d2-996e-858b7c03405a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:33.752465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28f95f-3ee3-4a2e-95d0-ce4f56851911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.753845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28f95f-3ee3-4a2e-95d0-ce4f56851911 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.754592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28f95f-3ee3-4a2e-95d0-ce4f56851911 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.755345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b28f95f-3ee3-4a2e-95d0-ce4f56851911 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.757339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28f95f-3ee3-4a2e-95d0-ce4f56851911 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:33.758033 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28f95f-3ee3-4a2e-95d0-ce4f56851911 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:33.759908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b28f95f-3ee3-4a2e-95d0-ce4f56851911 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:33.774104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:33.784323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 420/847] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Aug 1 11:51:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:33.790920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28f95f-3ee3-4a2e-95d0-ce4f56851911 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:33.851830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134e7cb-814a-46d2-996e-858b7c03405a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:33.890159 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88de94-2725-42de-a45c-944eec9b5a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.891121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88de94-2725-42de-a45c-944eec9b5a1e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.891571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88de94-2725-42de-a45c-944eec9b5a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.892039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88de94-2725-42de-a45c-944eec9b5a1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:33.903432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88de94-2725-42de-a45c-944eec9b5a1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-ea88de94-2725-42de-a45c-944eec9b5a1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:33.908842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:33.909843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:51:33.910663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:33.913531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 428/848] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:33.919473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88de94-2725-42de-a45c-944eec9b5a1e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:33.923023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b28f95f-3ee3-4a2e-95d0-ce4f56851911 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:33.925186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:33.962643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfdc04-a362-4fb6-81ff-4bb07590386e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:33.963501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfdc04-a362-4fb6-81ff-4bb07590386e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:33.963884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfdc04-a362-4fb6-81ff-4bb07590386e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:33.964257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfdc04-a362-4fb6-81ff-4bb07590386e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:34.040699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:34.045315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfdc04-a362-4fb6-81ff-4bb07590386e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:34.046543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88de94-2725-42de-a45c-944eec9b5a1e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:34.047302 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfdc04-a362-4fb6-81ff-4bb07590386e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:34.092463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceff56d-c5ed-42c6-b377-bb7827ad1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:34.093733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceff56d-c5ed-42c6-b377-bb7827ad1c49 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:34.094113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceff56d-c5ed-42c6-b377-bb7827ad1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:34.094487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceff56d-c5ed-42c6-b377-bb7827ad1c49 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:34.095728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceff56d-c5ed-42c6-b377-bb7827ad1c49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:34.096134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceff56d-c5ed-42c6-b377-bb7827ad1c49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:34.097990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceff56d-c5ed-42c6-b377-bb7827ad1c49 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:34.114504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 429/849] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:34.115877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceff56d-c5ed-42c6-b377-bb7827ad1c49 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:34.168935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:34.176007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 421/850] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 11:51:34.180570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfdc04-a362-4fb6-81ff-4bb07590386e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:34.226263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceff56d-c5ed-42c6-b377-bb7827ad1c49 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:34.263202 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2519f-5c02-4a23-8be0-5c6e156ec5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:34.264053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2519f-5c02-4a23-8be0-5c6e156ec5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:34.264462 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2519f-5c02-4a23-8be0-5c6e156ec5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:34.264814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2519f-5c02-4a23-8be0-5c6e156ec5f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:34.276737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2519f-5c02-4a23-8be0-5c6e156ec5f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:34.278574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2519f-5c02-4a23-8be0-5c6e156ec5f5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:34.284409 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfdc04-a362-4fb6-81ff-4bb07590386e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-dce2519f-5c02-4a23-8be0-5c6e156ec5f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:34.294127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:34.295255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:34.296386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:34.296386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:34.296386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:34.296386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:34.297516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 430/851] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:34.302978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2519f-5c02-4a23-8be0-5c6e156ec5f5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:34.320862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:34.329841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa94832-cf20-4762-ab68-4283c7ba20ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:34.330617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94832-cf20-4762-ab68-4283c7ba20ed None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:34.330979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94832-cf20-4762-ab68-4283c7ba20ed None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:34.331258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa94832-cf20-4762-ab68-4283c7ba20ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:34.342666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa94832-cf20-4762-ab68-4283c7ba20ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:34.344342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa94832-cf20-4762-ab68-4283c7ba20ed None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-5aa94832-cf20-4762-ab68-4283c7ba20ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:34.360911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:34.361845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:34.362869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:34.362869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:34.362869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:34.362869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:34.367576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 422/852] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:34.368595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa94832-cf20-4762-ab68-4283c7ba20ed None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:34.418922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2519f-5c02-4a23-8be0-5c6e156ec5f5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:34.425238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:34.466523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138914d-0089-49ec-b86c-774582566b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:34.467707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138914d-0089-49ec-b86c-774582566b82 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:34.468195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138914d-0089-49ec-b86c-774582566b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:34.468905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e138914d-0089-49ec-b86c-774582566b82 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:34.470264 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138914d-0089-49ec-b86c-774582566b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:34.470747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138914d-0089-49ec-b86c-774582566b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:34.472471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e138914d-0089-49ec-b86c-774582566b82 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:34.485979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa94832-cf20-4762-ab68-4283c7ba20ed None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:34.490647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 431/853] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:34.496662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138914d-0089-49ec-b86c-774582566b82 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:34.527765 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb64af0-4a0d-4d3f-81e3-80cd7dcb8a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:34.528457 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb64af0-4a0d-4d3f-81e3-80cd7dcb8a64 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:34.529003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb64af0-4a0d-4d3f-81e3-80cd7dcb8a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:34.529314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb64af0-4a0d-4d3f-81e3-80cd7dcb8a64 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:34.530021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb64af0-4a0d-4d3f-81e3-80cd7dcb8a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:34.530324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb64af0-4a0d-4d3f-81e3-80cd7dcb8a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:34.531718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb64af0-4a0d-4d3f-81e3-80cd7dcb8a64 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:34.549047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 423/854] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:34.550021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb64af0-4a0d-4d3f-81e3-80cd7dcb8a64 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:34.599203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:34.632037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e138914d-0089-49ec-b86c-774582566b82 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:34.671649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb64af0-4a0d-4d3f-81e3-80cd7dcb8a64 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:34.679064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bd9b6-490e-4bc9-88c0-e0889568f35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:34.680087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bd9b6-490e-4bc9-88c0-e0889568f35c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:34.680589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bd9b6-490e-4bc9-88c0-e0889568f35c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:34.681688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bd9b6-490e-4bc9-88c0-e0889568f35c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:34.685564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:34.694990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bd9b6-490e-4bc9-88c0-e0889568f35c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:34.696876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bd9b6-490e-4bc9-88c0-e0889568f35c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-e74bd9b6-490e-4bc9-88c0-e0889568f35c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:34.712785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:34.714178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:34.715114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:34.715114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:34.715114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:34.715114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:34.715114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:34.715114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:34.715114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:34.715114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:34.715114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 01 11:51:34.715114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:34.717542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 432/855] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:51:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:34.720860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28856a-26c0-4019-8f2f-c5c3b3aeef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:34.721751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28856a-26c0-4019-8f2f-c5c3b3aeef57 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:34.722151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28856a-26c0-4019-8f2f-c5c3b3aeef57 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:34.723112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bd9b6-490e-4bc9-88c0-e0889568f35c None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:34.723767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28856a-26c0-4019-8f2f-c5c3b3aeef57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:34.734807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28856a-26c0-4019-8f2f-c5c3b3aeef57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:34.736705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28856a-26c0-4019-8f2f-c5c3b3aeef57 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-1a28856a-26c0-4019-8f2f-c5c3b3aeef57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:34.750672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:34.751650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:34.753122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:34.753122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:34.753122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:34.753122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:34.753122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:34.753122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:34.753122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:34.753122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:34.761443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 424/856] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:34.763410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28856a-26c0-4019-8f2f-c5c3b3aeef57 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:34.843994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bd9b6-490e-4bc9-88c0-e0889568f35c None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:34.879603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:34.886402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28856a-26c0-4019-8f2f-c5c3b3aeef57 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:34.887785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429e6dd-af00-4386-a93c-cb3992a07de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:34.888777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429e6dd-af00-4386-a93c-cb3992a07de2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:34.889277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429e6dd-af00-4386-a93c-cb3992a07de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:34.889783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429e6dd-af00-4386-a93c-cb3992a07de2 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:34.890816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429e6dd-af00-4386-a93c-cb3992a07de2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:34.891323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429e6dd-af00-4386-a93c-cb3992a07de2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:34.893248 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429e6dd-af00-4386-a93c-cb3992a07de2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:34.918068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 433/857] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Aug 1 11:51:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:34.919554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429e6dd-af00-4386-a93c-cb3992a07de2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:34.928303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b3486-999b-4e6b-844a-528eef94dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:34.928968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b3486-999b-4e6b-844a-528eef94dc1d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:34.929249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b3486-999b-4e6b-844a-528eef94dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:34.929522 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b3486-999b-4e6b-844a-528eef94dc1d None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:34.930824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b3486-999b-4e6b-844a-528eef94dc1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:34.931139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b3486-999b-4e6b-844a-528eef94dc1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:34.933129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b3486-999b-4e6b-844a-528eef94dc1d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:34.950363 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 425/858] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:34.957007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b3486-999b-4e6b-844a-528eef94dc1d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:34.996896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:35.048891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429e6dd-af00-4386-a93c-cb3992a07de2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:35.073500 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b3486-999b-4e6b-844a-528eef94dc1d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:35.093399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e130f3-bf90-4f62-be33-f6f73f8c85c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:35.094395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e130f3-bf90-4f62-be33-f6f73f8c85c5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.094395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e130f3-bf90-4f62-be33-f6f73f8c85c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.094741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e130f3-bf90-4f62-be33-f6f73f8c85c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.120933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460e0f0-03c4-4db9-b847-aa7ca654ef8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:35.122467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460e0f0-03c4-4db9-b847-aa7ca654ef8b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.122927 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460e0f0-03c4-4db9-b847-aa7ca654ef8b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.123603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460e0f0-03c4-4db9-b847-aa7ca654ef8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.124872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460e0f0-03c4-4db9-b847-aa7ca654ef8b None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=bf76a8ec03e444e7b42e2997b8f73808) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:35.125785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460e0f0-03c4-4db9-b847-aa7ca654ef8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:35.127994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460e0f0-03c4-4db9-b847-aa7ca654ef8b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:35.136850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 426/859] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:34 2022] GET /identity/v3/role_assignments?user.id=bf76a8ec03e444e7b42e2997b8f73808&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:35.169585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:35.182116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e130f3-bf90-4f62-be33-f6f73f8c85c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:35.183971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e130f3-bf90-4f62-be33-f6f73f8c85c5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:35.301422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 434/860] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:51:35.304013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460e0f0-03c4-4db9-b847-aa7ca654ef8b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:35.372920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369cf52-f57d-45dc-9b75-f9da30640818 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.373208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369cf52-f57d-45dc-9b75-f9da30640818 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.373496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369cf52-f57d-45dc-9b75-f9da30640818 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.374868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 435/861] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:35.380090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87b8cb-fea5-4c50-85d1-de8899b212eb None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.380408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87b8cb-fea5-4c50-85d1-de8899b212eb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.380689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87b8cb-fea5-4c50-85d1-de8899b212eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.405761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3a87b8cb-fea5-4c50-85d1-de8899b212eb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:35.407344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3a87b8cb-fea5-4c50-85d1-de8899b212eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:35.423008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:35.433379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460e0f0-03c4-4db9-b847-aa7ca654ef8b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:35.470475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e2845-a8b5-4249-9640-11bae3829a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:35.471144 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e2845-a8b5-4249-9640-11bae3829a8f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.471475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e2845-a8b5-4249-9640-11bae3829a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.471752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e2845-a8b5-4249-9640-11bae3829a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.489464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e2845-a8b5-4249-9640-11bae3829a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:35.492101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e2845-a8b5-4249-9640-11bae3829a8f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:35.495824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 436/862] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-507e2845-a8b5-4249-9640-11bae3829a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:35.507846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:35.508980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:35.510103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:35.510103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:35.510103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:35.510103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:35.511976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 427/863] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:35.517576 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87b8cb-fea5-4c50-85d1-de8899b212eb None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:35.523105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e2845-a8b5-4249-9640-11bae3829a8f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:35.540012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:35.630573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87b8cb-fea5-4c50-85d1-de8899b212eb None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:35.637525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e2845-a8b5-4249-9640-11bae3829a8f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:35.673616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed5be2b-a408-4dfe-920f-d6b6bdcefd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:35.674301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5be2b-a408-4dfe-920f-d6b6bdcefd7f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.674550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5be2b-a408-4dfe-920f-d6b6bdcefd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.674651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ff873-d77d-47b7-b078-0ae3a7b86ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:35.674822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed5be2b-a408-4dfe-920f-d6b6bdcefd7f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.675185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ff873-d77d-47b7-b078-0ae3a7b86ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.675499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5be2b-a408-4dfe-920f-d6b6bdcefd7f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:35.675612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ff873-d77d-47b7-b078-0ae3a7b86ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.675693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ff873-d77d-47b7-b078-0ae3a7b86ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.676406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ff873-d77d-47b7-b078-0ae3a7b86ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:35.676715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ff873-d77d-47b7-b078-0ae3a7b86ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:35.676807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed5be2b-a408-4dfe-920f-d6b6bdcefd7f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:35.678080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ff873-d77d-47b7-b078-0ae3a7b86ad8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-7ed5be2b-a408-4dfe-920f-d6b6bdcefd7f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:35.686001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:35.687202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:35.688046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 428/864] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Aug 1 11:51:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:35.689384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738b9de-1475-492d-a1af-10ea0795f476 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.689384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738b9de-1475-492d-a1af-10ea0795f476 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.689384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738b9de-1475-492d-a1af-10ea0795f476 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.690754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 429/865] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:35.693415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738b9de-1475-492d-a1af-10ea0795f476 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:35.695231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 437/866] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:35.696554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1907d-94e3-4d70-9a46-d25c17f73bdc None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.696842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1907d-94e3-4d70-9a46-d25c17f73bdc None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.697119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1907d-94e3-4d70-9a46-d25c17f73bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.708757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:35.715990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-38b1907d-94e3-4d70-9a46-d25c17f73bdc None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:35.717231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-38b1907d-94e3-4d70-9a46-d25c17f73bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:35.791307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 438/867] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:35.792182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1907d-94e3-4d70-9a46-d25c17f73bdc None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:35.811714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:35.818027 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738b9de-1475-492d-a1af-10ea0795f476 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:35.863121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52780-42ab-4ee5-962b-956da1686530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:35.863744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52780-42ab-4ee5-962b-956da1686530 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.864060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52780-42ab-4ee5-962b-956da1686530 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.864360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab52780-42ab-4ee5-962b-956da1686530 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.864984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab52780-42ab-4ee5-962b-956da1686530 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:35.865286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab52780-42ab-4ee5-962b-956da1686530 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:35.866664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab52780-42ab-4ee5-962b-956da1686530 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:35.875788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 430/868] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:35.875788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52780-42ab-4ee5-962b-956da1686530 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:35.919884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1907d-94e3-4d70-9a46-d25c17f73bdc None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:35.955942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:35.956878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ef974-24be-4d3e-b574-23ad623a7088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:35.957589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ef974-24be-4d3e-b574-23ad623a7088 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:35.957927 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ef974-24be-4d3e-b574-23ad623a7088 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:35.958251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ef974-24be-4d3e-b574-23ad623a7088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:35.966012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ef974-24be-4d3e-b574-23ad623a7088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:35.967918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ef974-24be-4d3e-b574-23ad623a7088 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-416ef974-24be-4d3e-b574-23ad623a7088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:35.982992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:35.984272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:35.990456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:35.990456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:35.990456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:35.990456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:35.990456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:35.990456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:35.990456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:35.990456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:35.990906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab52780-42ab-4ee5-962b-956da1686530 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:35.990993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 439/869] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:35.990993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ef974-24be-4d3e-b574-23ad623a7088 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:36.020610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d4fe7-12c1-4ff6-914d-4690284a1264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:36.021226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d4fe7-12c1-4ff6-914d-4690284a1264 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:36.021505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d4fe7-12c1-4ff6-914d-4690284a1264 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:36.021775 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d4fe7-12c1-4ff6-914d-4690284a1264 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:36.022465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d4fe7-12c1-4ff6-914d-4690284a1264 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:36.023847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92d4fe7-12c1-4ff6-914d-4690284a1264 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:36.043567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 431/870] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:51:35 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:36.045702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d4fe7-12c1-4ff6-914d-4690284a1264 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:36.074099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:36.116173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ef974-24be-4d3e-b574-23ad623a7088 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:36.164183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01adde7-8dc6-4e3b-b3e0-9b1463c12de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:36.165046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01adde7-8dc6-4e3b-b3e0-9b1463c12de4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:36.165407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01adde7-8dc6-4e3b-b3e0-9b1463c12de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:36.165749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01adde7-8dc6-4e3b-b3e0-9b1463c12de4 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:36.166834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01adde7-8dc6-4e3b-b3e0-9b1463c12de4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:36.167235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01adde7-8dc6-4e3b-b3e0-9b1463c12de4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:36.168882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01adde7-8dc6-4e3b-b3e0-9b1463c12de4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:36.180466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 440/871] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:35 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:36.204318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92d4fe7-12c1-4ff6-914d-4690284a1264 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:36.250399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d5d04-1e77-41da-9a6a-353d9a7d125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:36.251358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d5d04-1e77-41da-9a6a-353d9a7d125e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:36.251635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d5d04-1e77-41da-9a6a-353d9a7d125e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:36.251906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d5d04-1e77-41da-9a6a-353d9a7d125e None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:36.252768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d5d04-1e77-41da-9a6a-353d9a7d125e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:36.253165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d5d04-1e77-41da-9a6a-353d9a7d125e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:36.254642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d5d04-1e77-41da-9a6a-353d9a7d125e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:36.258341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:36.270221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 432/872] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:36.276366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01adde7-8dc6-4e3b-b3e0-9b1463c12de4 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:36.390222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01adde7-8dc6-4e3b-b3e0-9b1463c12de4 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:36.420382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:36.427446 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cf35b-31e0-4001-9b2e-a04bf3748f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:36.428072 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cf35b-31e0-4001-9b2e-a04bf3748f0c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:36.428372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cf35b-31e0-4001-9b2e-a04bf3748f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:36.428775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cf35b-31e0-4001-9b2e-a04bf3748f0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:36.429631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cf35b-31e0-4001-9b2e-a04bf3748f0c None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=b9b6a96421e349a88a38590cfce167fb) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:36.429939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cf35b-31e0-4001-9b2e-a04bf3748f0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:36.431391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cf35b-31e0-4001-9b2e-a04bf3748f0c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:36.438188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 441/873] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:36 2022] GET /identity/v3/role_assignments?user.id=b9b6a96421e349a88a38590cfce167fb&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:37.283222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10c272-74a6-4801-abb7-ef4719861285 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:37.284342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10c272-74a6-4801-abb7-ef4719861285 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:37.284816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10c272-74a6-4801-abb7-ef4719861285 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:37.286656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 433/874] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:37.292322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d091c5-289c-4747-a12e-e6aff5ffadd7 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:37.293140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d091c5-289c-4747-a12e-e6aff5ffadd7 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:37.293604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d091c5-289c-4747-a12e-e6aff5ffadd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:37.312009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-34d091c5-289c-4747-a12e-e6aff5ffadd7 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:37.313543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-34d091c5-289c-4747-a12e-e6aff5ffadd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:37.388443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 442/875] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:37 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:37.407022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10c272-74a6-4801-abb7-ef4719861285 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:37.520765 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10c272-74a6-4801-abb7-ef4719861285 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:37.556936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645340ed-5968-4122-9337-3c7fbbecec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:37.558049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645340ed-5968-4122-9337-3c7fbbecec2d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:37.558541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645340ed-5968-4122-9337-3c7fbbecec2d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:37.559009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645340ed-5968-4122-9337-3c7fbbecec2d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:37.572125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645340ed-5968-4122-9337-3c7fbbecec2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-645340ed-5968-4122-9337-3c7fbbecec2d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:37.577365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:37.578480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:51:37.579508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:37.586527 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 434/876] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Aug 1 11:51:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:37.588053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d091c5-289c-4747-a12e-e6aff5ffadd7 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:37.686409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d091c5-289c-4747-a12e-e6aff5ffadd7 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:37.710810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:37.722339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc1fef-11c0-4526-9a2e-600a7b34b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:37.722964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc1fef-11c0-4526-9a2e-600a7b34b1dd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:37.723241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc1fef-11c0-4526-9a2e-600a7b34b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:37.723513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc1fef-11c0-4526-9a2e-600a7b34b1dd None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:37.724237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc1fef-11c0-4526-9a2e-600a7b34b1dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:37.724583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc1fef-11c0-4526-9a2e-600a7b34b1dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:37.725967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc1fef-11c0-4526-9a2e-600a7b34b1dd None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:37.740035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 443/877] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Aug 1 11:51:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:37.741422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645340ed-5968-4122-9337-3c7fbbecec2d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:37.805050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:37.859909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645340ed-5968-4122-9337-3c7fbbecec2d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:37.896118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91e3e5-59dc-45ff-b682-a9657ef8d530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:37.896916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91e3e5-59dc-45ff-b682-a9657ef8d530 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:37.897197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91e3e5-59dc-45ff-b682-a9657ef8d530 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:37.897470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91e3e5-59dc-45ff-b682-a9657ef8d530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:37.908886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91e3e5-59dc-45ff-b682-a9657ef8d530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:37.910385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91e3e5-59dc-45ff-b682-a9657ef8d530 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-7f91e3e5-59dc-45ff-b682-a9657ef8d530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:37.925521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:37.926434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:37.927314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:37.927314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:37.927314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:37.927314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:37.931452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 435/878] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:37.932861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc1fef-11c0-4526-9a2e-600a7b34b1dd None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:38.037204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc1fef-11c0-4526-9a2e-600a7b34b1dd None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:38.072274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262de66-3943-4b79-b0af-5c010f422a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:38.072987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262de66-3943-4b79-b0af-5c010f422a7c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.073269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262de66-3943-4b79-b0af-5c010f422a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.073536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262de66-3943-4b79-b0af-5c010f422a7c None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.074230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262de66-3943-4b79-b0af-5c010f422a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:38.074529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262de66-3943-4b79-b0af-5c010f422a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:38.076007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262de66-3943-4b79-b0af-5c010f422a7c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:38.092451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 444/879] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:38.094773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91e3e5-59dc-45ff-b682-a9657ef8d530 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:38.188409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ee567-e9be-40ab-aac7-dff620f649e3 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.188585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ee567-e9be-40ab-aac7-dff620f649e3 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.188859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ee567-e9be-40ab-aac7-dff620f649e3 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.191354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 445/880] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:51:38.200285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a0955-52ee-4552-aed9-0a2dabe496ec None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.200875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a0955-52ee-4552-aed9-0a2dabe496ec None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.201345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a0955-52ee-4552-aed9-0a2dabe496ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.207412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91e3e5-59dc-45ff-b682-a9657ef8d530 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:38.217852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:38.221301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4f6a0955-52ee-4552-aed9-0a2dabe496ec None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:38.222862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4f6a0955-52ee-4552-aed9-0a2dabe496ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:38.257369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5bbc3-167a-48bb-b84d-ee50898e8194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:38.258127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5bbc3-167a-48bb-b84d-ee50898e8194 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.258405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5bbc3-167a-48bb-b84d-ee50898e8194 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.258674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5bbc3-167a-48bb-b84d-ee50898e8194 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.267681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5bbc3-167a-48bb-b84d-ee50898e8194 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:38.269314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5bbc3-167a-48bb-b84d-ee50898e8194 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:38.269795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-a3c5bbc3-167a-48bb-b84d-ee50898e8194 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:38.284588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:38.285990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:38.286931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 436/881] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Aug 1 11:51:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:38.287553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413a16e-2812-408c-bdb9-fdd6f123a1e7 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.287553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413a16e-2812-408c-bdb9-fdd6f123a1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.288183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413a16e-2812-408c-bdb9-fdd6f123a1e7 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.289129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 437/882] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:38.294780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d49156-1343-4f27-9069-82861ee564a8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.295064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d49156-1343-4f27-9069-82861ee564a8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.295334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d49156-1343-4f27-9069-82861ee564a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.301021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 446/883] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:38 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:38.307449 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a0955-52ee-4552-aed9-0a2dabe496ec None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:38.313831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-90d49156-1343-4f27-9069-82861ee564a8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:38.315071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-90d49156-1343-4f27-9069-82861ee564a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:38.398823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 438/884] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:38 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:38.398823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d49156-1343-4f27-9069-82861ee564a8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:38.412162 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a0955-52ee-4552-aed9-0a2dabe496ec None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:38.447202 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4750a9-ffb7-42df-90c5-2deded8f2159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:38.447739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4750a9-ffb7-42df-90c5-2deded8f2159 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.448015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4750a9-ffb7-42df-90c5-2deded8f2159 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.448285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4750a9-ffb7-42df-90c5-2deded8f2159 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.449117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4750a9-ffb7-42df-90c5-2deded8f2159 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:38.449424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4750a9-ffb7-42df-90c5-2deded8f2159 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:38.450864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4750a9-ffb7-42df-90c5-2deded8f2159 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:38.468748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 447/885] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Aug 1 11:51:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:38.469533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4750a9-ffb7-42df-90c5-2deded8f2159 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:38.505889 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:38.510805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:38.517357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d49156-1343-4f27-9069-82861ee564a8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:38.559726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf0d74-cdc2-413f-a34b-a8d53d0b1f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:38.560399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf0d74-cdc2-413f-a34b-a8d53d0b1f34 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.560685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf0d74-cdc2-413f-a34b-a8d53d0b1f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.561048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf0d74-cdc2-413f-a34b-a8d53d0b1f34 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.563081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf0d74-cdc2-413f-a34b-a8d53d0b1f34 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:38.563081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf0d74-cdc2-413f-a34b-a8d53d0b1f34 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-a3bf0d74-cdc2-413f-a34b-a8d53d0b1f34 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:38.572497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:38.573776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 11:51:38.574793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:38.575733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4750a9-ffb7-42df-90c5-2deded8f2159 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:38.575819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 439/886] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:51:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:38.575819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21bcca-d9d4-45b2-aaea-2016c04f0cba None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.575819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21bcca-d9d4-45b2-aaea-2016c04f0cba None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.575819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21bcca-d9d4-45b2-aaea-2016c04f0cba None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.576833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 440/887] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:38.577420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21bcca-d9d4-45b2-aaea-2016c04f0cba None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:38.615582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61844dd-2712-429a-b24c-91f70511ad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:38.616206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61844dd-2712-429a-b24c-91f70511ad8e None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.616513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61844dd-2712-429a-b24c-91f70511ad8e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.616784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61844dd-2712-429a-b24c-91f70511ad8e None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.617482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61844dd-2712-429a-b24c-91f70511ad8e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:38.618861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61844dd-2712-429a-b24c-91f70511ad8e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:38.634193 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 448/888] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:51:38 2022] POST /identity/v3/services => generated 269 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:38.635093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61844dd-2712-429a-b24c-91f70511ad8e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:38.709236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21bcca-d9d4-45b2-aaea-2016c04f0cba None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:38.746320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69e0eb-c845-4eae-803c-52969b388905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:38.746958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69e0eb-c845-4eae-803c-52969b388905 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.747236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69e0eb-c845-4eae-803c-52969b388905 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.747494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69e0eb-c845-4eae-803c-52969b388905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.754448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:38.764940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61844dd-2712-429a-b24c-91f70511ad8e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:38.779211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:38.811377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85534ad-45e4-47e4-9632-4d9cacaba0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:38.812431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85534ad-45e4-47e4-9632-4d9cacaba0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.813389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85534ad-45e4-47e4-9632-4d9cacaba0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.813389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85534ad-45e4-47e4-9632-4d9cacaba0d0 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.813777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85534ad-45e4-47e4-9632-4d9cacaba0d0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:38.814691 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85534ad-45e4-47e4-9632-4d9cacaba0d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:38.815581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85534ad-45e4-47e4-9632-4d9cacaba0d0 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:38.824067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 449/889] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:51:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:38.825455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d70bea-4a44-43ab-8d4f-8e10d9522bea None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:38.825565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d70bea-4a44-43ab-8d4f-8e10d9522bea None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:38.825802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d70bea-4a44-43ab-8d4f-8e10d9522bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:38.832394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69e0eb-c845-4eae-803c-52969b388905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:38.833990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69e0eb-c845-4eae-803c-52969b388905 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:38.850868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-30d70bea-4a44-43ab-8d4f-8e10d9522bea None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:38.852060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-30d70bea-4a44-43ab-8d4f-8e10d9522bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:38.922173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 450/890] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:38.923131 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d70bea-4a44-43ab-8d4f-8e10d9522bea None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:38.948620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 441/891] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:38 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:51:38.949481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69e0eb-c845-4eae-803c-52969b388905 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:39.009899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:39.024272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:39.038070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d70bea-4a44-43ab-8d4f-8e10d9522bea None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:39.058545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69e0eb-c845-4eae-803c-52969b388905 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:39.072839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26835393-ccf7-41be-8a2c-b667db9ef701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:39.073473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26835393-ccf7-41be-8a2c-b667db9ef701 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:39.073749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26835393-ccf7-41be-8a2c-b667db9ef701 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:39.074022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26835393-ccf7-41be-8a2c-b667db9ef701 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:39.074707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26835393-ccf7-41be-8a2c-b667db9ef701 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:39.075012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26835393-ccf7-41be-8a2c-b667db9ef701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:39.076424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26835393-ccf7-41be-8a2c-b667db9ef701 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:39.086746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 451/892] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:51:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:39.086746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26835393-ccf7-41be-8a2c-b667db9ef701 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:39.098867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea3aa3-116d-4f38-bca0-2fde9e71cf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:39.099689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea3aa3-116d-4f38-bca0-2fde9e71cf5e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:39.099798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea3aa3-116d-4f38-bca0-2fde9e71cf5e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:39.100098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea3aa3-116d-4f38-bca0-2fde9e71cf5e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:39.100773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea3aa3-116d-4f38-bca0-2fde9e71cf5e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:39.102212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aea3aa3-116d-4f38-bca0-2fde9e71cf5e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-1aea3aa3-116d-4f38-bca0-2fde9e71cf5e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:39.112032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:39.113095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:39.113951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 442/893] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:39.114720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea3aa3-116d-4f38-bca0-2fde9e71cf5e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:39.196364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26835393-ccf7-41be-8a2c-b667db9ef701 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:39.225736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aea3aa3-116d-4f38-bca0-2fde9e71cf5e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:39.231058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9816294-48fd-4916-9818-2423ca526ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:39.231704 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816294-48fd-4916-9818-2423ca526ebe None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:39.231976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816294-48fd-4916-9818-2423ca526ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:39.232246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9816294-48fd-4916-9818-2423ca526ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:39.242726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9816294-48fd-4916-9818-2423ca526ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:39.244179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9816294-48fd-4916-9818-2423ca526ebe None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:39.248250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-d9816294-48fd-4916-9818-2423ca526ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:39.259160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:39.260333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:39.261260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:39.261260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:39.261260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:39.261260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:39.261260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 452/894] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:39.261532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9816294-48fd-4916-9818-2423ca526ebe None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:39.269224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:39.269477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea31dc-c441-433b-b8c3-294268c48ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:39.270115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea31dc-c441-433b-b8c3-294268c48ebf None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:39.270641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea31dc-c441-433b-b8c3-294268c48ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:39.270641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea31dc-c441-433b-b8c3-294268c48ebf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:39.271386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea31dc-c441-433b-b8c3-294268c48ebf None admin] RBAC: Adding query filter params (interface=public, service_id=9558da7a4ebc41edac9729b30456c938, region_id=RegionOne) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:39.271701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea31dc-c441-433b-b8c3-294268c48ebf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:39.273119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea31dc-c441-433b-b8c3-294268c48ebf None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:39.279780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 443/895] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Aug 1 11:51:39 2022] GET /identity/v3/endpoints?service_id=9558da7a4ebc41edac9729b30456c938&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:39.280389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea31dc-c441-433b-b8c3-294268c48ebf None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:39.380146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9816294-48fd-4916-9818-2423ca526ebe None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:39.392255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea31dc-c441-433b-b8c3-294268c48ebf None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:39.416733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d75b8-915d-4eb2-b621-0fcaa2b842be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:39.417382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d75b8-915d-4eb2-b621-0fcaa2b842be None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:39.417663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d75b8-915d-4eb2-b621-0fcaa2b842be None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:39.417934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d75b8-915d-4eb2-b621-0fcaa2b842be None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:39.418602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d75b8-915d-4eb2-b621-0fcaa2b842be None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:39.419006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d75b8-915d-4eb2-b621-0fcaa2b842be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:39.420312 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d75b8-915d-4eb2-b621-0fcaa2b842be None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:39.430456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dd683-8aaa-48c4-a6f3-9402d2774a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:39.430965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 453/896] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Aug 1 11:51:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:39.431091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dd683-8aaa-48c4-a6f3-9402d2774a95 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:39.431375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dd683-8aaa-48c4-a6f3-9402d2774a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:39.431766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1dd683-8aaa-48c4-a6f3-9402d2774a95 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:39.431974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d75b8-915d-4eb2-b621-0fcaa2b842be None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:39.432491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1dd683-8aaa-48c4-a6f3-9402d2774a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:39.432802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1dd683-8aaa-48c4-a6f3-9402d2774a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:39.434172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1dd683-8aaa-48c4-a6f3-9402d2774a95 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:39.447099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 444/897] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:39.447910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dd683-8aaa-48c4-a6f3-9402d2774a95 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:39.496711 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:39.509786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:39.548749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d75b8-915d-4eb2-b621-0fcaa2b842be None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:39.557340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1dd683-8aaa-48c4-a6f3-9402d2774a95 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:39.584226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceadeb4-2665-485f-a7c2-7d392580dfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:39.585097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceadeb4-2665-485f-a7c2-7d392580dfbc None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:39.586538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceadeb4-2665-485f-a7c2-7d392580dfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:39.586538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceadeb4-2665-485f-a7c2-7d392580dfbc None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:39.587143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceadeb4-2665-485f-a7c2-7d392580dfbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:39.589820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceadeb4-2665-485f-a7c2-7d392580dfbc None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:39.595216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf45099-ef92-4319-8288-88d88415737b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:39.595846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf45099-ef92-4319-8288-88d88415737b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:39.596182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf45099-ef92-4319-8288-88d88415737b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:39.596486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf45099-ef92-4319-8288-88d88415737b None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:39.597294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf45099-ef92-4319-8288-88d88415737b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:39.598091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf45099-ef92-4319-8288-88d88415737b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:39.599220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf45099-ef92-4319-8288-88d88415737b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:39.599419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 454/898] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Aug 1 11:51:39 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 11:51:39.604347 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 445/899] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Aug 1 11:51:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:39.616540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceadeb4-2665-485f-a7c2-7d392580dfbc None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:39.741246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceadeb4-2665-485f-a7c2-7d392580dfbc None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:39.779039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:39.787684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21d554-58ea-4987-9b61-1aba62883e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:39.788806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21d554-58ea-4987-9b61-1aba62883e58 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:39.789298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21d554-58ea-4987-9b61-1aba62883e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:39.789773 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21d554-58ea-4987-9b61-1aba62883e58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:39.798984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21d554-58ea-4987-9b61-1aba62883e58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:39.803282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21d554-58ea-4987-9b61-1aba62883e58 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-2f21d554-58ea-4987-9b61-1aba62883e58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:39.815348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:39.816535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:39.817546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:39.817546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:39.817546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:39.817546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:39.817546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:39.817546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:39.817546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:39.817546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:39.817546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 455/900] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:39.822825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf45099-ef92-4319-8288-88d88415737b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:39.933733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf45099-ef92-4319-8288-88d88415737b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:39.943840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:39.973805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd55b0d-ee4b-467d-bc03-b24efe65736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:39.974749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd55b0d-ee4b-467d-bc03-b24efe65736e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:39.975228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd55b0d-ee4b-467d-bc03-b24efe65736e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:39.975674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd55b0d-ee4b-467d-bc03-b24efe65736e None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:39.976637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd55b0d-ee4b-467d-bc03-b24efe65736e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:39.977141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd55b0d-ee4b-467d-bc03-b24efe65736e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:39.978752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd55b0d-ee4b-467d-bc03-b24efe65736e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:39.992388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 446/901] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:39.998696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21d554-58ea-4987-9b61-1aba62883e58 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:40.105348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21d554-58ea-4987-9b61-1aba62883e58 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:40.143283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efda09-a81b-4dee-ab6e-68c2aeb68df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:40.143909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efda09-a81b-4dee-ab6e-68c2aeb68df0 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:40.144187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efda09-a81b-4dee-ab6e-68c2aeb68df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:40.144567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efda09-a81b-4dee-ab6e-68c2aeb68df0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:40.145364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efda09-a81b-4dee-ab6e-68c2aeb68df0 None admin] RBAC: Adding query filter params (role.id=9a4d5a7314884a599e1b22288a1add9d, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=bf76a8ec03e444e7b42e2997b8f73808) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:40.145671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efda09-a81b-4dee-ab6e-68c2aeb68df0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:40.147179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efda09-a81b-4dee-ab6e-68c2aeb68df0 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:40.154533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 456/902] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:39 2022] GET /identity/v3/role_assignments?user.id=bf76a8ec03e444e7b42e2997b8f73808&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=9a4d5a7314884a599e1b22288a1add9d => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:40.681180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5ff33-712c-41d6-a501-0d6b319265a8 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:40.681713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5ff33-712c-41d6-a501-0d6b319265a8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:40.681713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5ff33-712c-41d6-a501-0d6b319265a8 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:40.683354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 447/903] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:40.688877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299baacf-aa8d-46ec-b65a-278c218d2224 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:40.689294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299baacf-aa8d-46ec-b65a-278c218d2224 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:40.689464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299baacf-aa8d-46ec-b65a-278c218d2224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:40.708476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-299baacf-aa8d-46ec-b65a-278c218d2224 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:40.709678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-299baacf-aa8d-46ec-b65a-278c218d2224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:40.764649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:40.784432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 457/904] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:40.806566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5ff33-712c-41d6-a501-0d6b319265a8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:40.930797 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5ff33-712c-41d6-a501-0d6b319265a8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:40.965568 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4a6ba-c63a-4a4f-8439-7a86e754fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:40.966140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4a6ba-c63a-4a4f-8439-7a86e754fa37 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:40.966419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4a6ba-c63a-4a4f-8439-7a86e754fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:40.966691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4a6ba-c63a-4a4f-8439-7a86e754fa37 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:40.967319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4a6ba-c63a-4a4f-8439-7a86e754fa37 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:40.968726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4a6ba-c63a-4a4f-8439-7a86e754fa37 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-b2d4a6ba-c63a-4a4f-8439-7a86e754fa37 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:40.977896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:40.978801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:40.979610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 448/905] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Aug 1 11:51:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:40.989395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299baacf-aa8d-46ec-b65a-278c218d2224 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:41.088520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299baacf-aa8d-46ec-b65a-278c218d2224 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:41.125092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22dffc-bcf8-45c7-8ced-212e84e878de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:41.125724 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22dffc-bcf8-45c7-8ced-212e84e878de None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:41.126002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22dffc-bcf8-45c7-8ced-212e84e878de None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:41.126276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb22dffc-bcf8-45c7-8ced-212e84e878de None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:41.126965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22dffc-bcf8-45c7-8ced-212e84e878de None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:41.127333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22dffc-bcf8-45c7-8ced-212e84e878de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:41.128838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb22dffc-bcf8-45c7-8ced-212e84e878de None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:41.134002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 458/906] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:41.139500 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4a6ba-c63a-4a4f-8439-7a86e754fa37 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:41.173012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:41.244852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4a6ba-c63a-4a4f-8439-7a86e754fa37 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:41.279802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb9f0e-c3eb-4400-9b81-8c80c19363e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:41.280468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb9f0e-c3eb-4400-9b81-8c80c19363e3 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:41.280747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb9f0e-c3eb-4400-9b81-8c80c19363e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:41.281010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb9f0e-c3eb-4400-9b81-8c80c19363e3 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:41.281714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb9f0e-c3eb-4400-9b81-8c80c19363e3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:41.282002 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb9f0e-c3eb-4400-9b81-8c80c19363e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:41.283390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb9f0e-c3eb-4400-9b81-8c80c19363e3 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:41.289180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 449/907] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:41.295009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22dffc-bcf8-45c7-8ced-212e84e878de None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:41.349026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:41.400958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb22dffc-bcf8-45c7-8ced-212e84e878de None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:41.436960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37892d-f65f-4df8-b4b1-f227005d965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:41.437594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37892d-f65f-4df8-b4b1-f227005d965f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:41.437921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37892d-f65f-4df8-b4b1-f227005d965f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:41.438198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37892d-f65f-4df8-b4b1-f227005d965f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:41.438962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37892d-f65f-4df8-b4b1-f227005d965f None admin] RBAC: Adding query filter params (interface=public, service_id=58a4d49905bc49549454cfe208522b77, region_id=RegionOne) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:41.439268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37892d-f65f-4df8-b4b1-f227005d965f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:41.440731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37892d-f65f-4df8-b4b1-f227005d965f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:41.446382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 459/908] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Aug 1 11:51:41 2022] GET /identity/v3/endpoints?service_id=58a4d49905bc49549454cfe208522b77&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:41.451881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb9f0e-c3eb-4400-9b81-8c80c19363e3 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:41.546863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:41.554984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb9f0e-c3eb-4400-9b81-8c80c19363e3 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:41.591848 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7967a7-27a2-44ff-a9bc-fee64b82571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:41.592507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7967a7-27a2-44ff-a9bc-fee64b82571c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:41.592791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7967a7-27a2-44ff-a9bc-fee64b82571c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:41.593062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7967a7-27a2-44ff-a9bc-fee64b82571c None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:41.593720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7967a7-27a2-44ff-a9bc-fee64b82571c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:41.595104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7967a7-27a2-44ff-a9bc-fee64b82571c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:41.601498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 450/909] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Aug 1 11:51:41 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 11:51:41.679493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b2adb-906d-4024-96ae-b8af467951f5 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:41.679834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b2adb-906d-4024-96ae-b8af467951f5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:41.680089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b2adb-906d-4024-96ae-b8af467951f5 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:41.681836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 460/910] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:41.687833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344e52f-76f5-4e66-a2a5-e5dc4881352d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:41.688420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344e52f-76f5-4e66-a2a5-e5dc4881352d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:41.688420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344e52f-76f5-4e66-a2a5-e5dc4881352d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:41.708354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3344e52f-76f5-4e66-a2a5-e5dc4881352d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:41.709606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3344e52f-76f5-4e66-a2a5-e5dc4881352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:41.741247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae34391-a51f-4f2f-9448-7963f8bba614 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:41.741770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae34391-a51f-4f2f-9448-7963f8bba614 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:41.741770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae34391-a51f-4f2f-9448-7963f8bba614 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:41.743395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 461/911] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:41.749432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a385b9-85e4-4500-be6b-bd538c49964a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:41.749805 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a385b9-85e4-4500-be6b-bd538c49964a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:41.750083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a385b9-85e4-4500-be6b-bd538c49964a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:41.770719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d4a385b9-85e4-4500-be6b-bd538c49964a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:41.772025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d4a385b9-85e4-4500-be6b-bd538c49964a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:41.792389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 451/912] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:41.817638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344e52f-76f5-4e66-a2a5-e5dc4881352d None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:41.860717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 462/913] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:41.887503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a385b9-85e4-4500-be6b-bd538c49964a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:41.933072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:41.938254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:41.947667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344e52f-76f5-4e66-a2a5-e5dc4881352d None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:41.984944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa29e0-bc87-4eb2-ae06-43d34e56f4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:41.985562 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa29e0-bc87-4eb2-ae06-43d34e56f4bd None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:41.985836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa29e0-bc87-4eb2-ae06-43d34e56f4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:41.986105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa29e0-bc87-4eb2-ae06-43d34e56f4bd None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:41.986933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa29e0-bc87-4eb2-ae06-43d34e56f4bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:41.988320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa29e0-bc87-4eb2-ae06-43d34e56f4bd None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:42.008719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a385b9-85e4-4500-be6b-bd538c49964a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:42.012337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 452/914] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:51:41 2022] POST /identity/v3/services => generated 241 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 11:51:42.067512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ca20f-54a6-4b88-a4c8-1b8106aa3a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:42.068245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ca20f-54a6-4b88-a4c8-1b8106aa3a4b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:42.068498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ca20f-54a6-4b88-a4c8-1b8106aa3a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:42.068772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ca20f-54a6-4b88-a4c8-1b8106aa3a4b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:42.069415 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ca20f-54a6-4b88-a4c8-1b8106aa3a4b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:42.070948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ca20f-54a6-4b88-a4c8-1b8106aa3a4b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-749ca20f-54a6-4b88-a4c8-1b8106aa3a4b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:42.081631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:42.082711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:42.083583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 463/915] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:51:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:42.091901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa29e0-bc87-4eb2-ae06-43d34e56f4bd None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:42.205604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa29e0-bc87-4eb2-ae06-43d34e56f4bd None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:42.243224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aecded-79e2-4bc5-9cbc-46b47e65ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:42.244241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecded-79e2-4bc5-9cbc-46b47e65ab6f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:42.244763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecded-79e2-4bc5-9cbc-46b47e65ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:42.245226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecded-79e2-4bc5-9cbc-46b47e65ab6f None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:42.246095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aecded-79e2-4bc5-9cbc-46b47e65ab6f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:42.246585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aecded-79e2-4bc5-9cbc-46b47e65ab6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:42.248155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aecded-79e2-4bc5-9cbc-46b47e65ab6f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:42.253900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 453/916] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:51:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:42.260568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ca20f-54a6-4b88-a4c8-1b8106aa3a4b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:42.279093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a40ca-8244-4a90-b73b-38b84eab30a3 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:42.279423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a40ca-8244-4a90-b73b-38b84eab30a3 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:42.279655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a40ca-8244-4a90-b73b-38b84eab30a3 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:42.282365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 454/917] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:51:42.286663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e826f90-0ab3-4636-8890-1694df09bdfc None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:42.286953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e826f90-0ab3-4636-8890-1694df09bdfc None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:42.287235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e826f90-0ab3-4636-8890-1694df09bdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:42.309268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1e826f90-0ab3-4636-8890-1694df09bdfc None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:42.310489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1e826f90-0ab3-4636-8890-1694df09bdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:42.324638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:42.382559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ca20f-54a6-4b88-a4c8-1b8106aa3a4b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:42.386487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:42.412392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 455/918] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:42.427717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e826f90-0ab3-4636-8890-1694df09bdfc None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:42.431014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2f581-27f2-499c-95eb-f44545fd9efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:42.431892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2f581-27f2-499c-95eb-f44545fd9efd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:42.432383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2f581-27f2-499c-95eb-f44545fd9efd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:42.432908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2f581-27f2-499c-95eb-f44545fd9efd None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:42.433814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2f581-27f2-499c-95eb-f44545fd9efd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:42.434317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2f581-27f2-499c-95eb-f44545fd9efd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:42.436017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f2f581-27f2-499c-95eb-f44545fd9efd None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:42.442220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 464/919] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Aug 1 11:51:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:42.448135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2f581-27f2-499c-95eb-f44545fd9efd None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:42.529525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e826f90-0ab3-4636-8890-1694df09bdfc None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:42.560100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f2f581-27f2-499c-95eb-f44545fd9efd None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:42.564912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c28c80-249d-4611-b228-32267c037a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:42.565601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c28c80-249d-4611-b228-32267c037a6c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:42.565879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c28c80-249d-4611-b228-32267c037a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:42.566151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c28c80-249d-4611-b228-32267c037a6c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:42.577914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c28c80-249d-4611-b228-32267c037a6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-c1c28c80-249d-4611-b228-32267c037a6c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:42.582932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:42.583904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:51:42.584764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:42.589145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 456/920] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Aug 1 11:51:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:42.590477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c28c80-249d-4611-b228-32267c037a6c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:42.595033 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:42.610830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c941d5c-e4e4-4f26-a4ad-059492e96ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:42.611890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c941d5c-e4e4-4f26-a4ad-059492e96ac4 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:42.612468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c941d5c-e4e4-4f26-a4ad-059492e96ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:42.612941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c941d5c-e4e4-4f26-a4ad-059492e96ac4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:42.613827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c941d5c-e4e4-4f26-a4ad-059492e96ac4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:42.615659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c941d5c-e4e4-4f26-a4ad-059492e96ac4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:42.644504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 465/921] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 11:51:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:42.710047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c28c80-249d-4611-b228-32267c037a6c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:42.755786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45941ed7-246a-481f-a00c-f1ce6a0d703e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:42.756461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45941ed7-246a-481f-a00c-f1ce6a0d703e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:42.756750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45941ed7-246a-481f-a00c-f1ce6a0d703e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:42.757024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45941ed7-246a-481f-a00c-f1ce6a0d703e None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:42.757761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45941ed7-246a-481f-a00c-f1ce6a0d703e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:42.758066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45941ed7-246a-481f-a00c-f1ce6a0d703e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:42.759486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45941ed7-246a-481f-a00c-f1ce6a0d703e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:42.774414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 457/922] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Aug 1 11:51:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:42.777863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c941d5c-e4e4-4f26-a4ad-059492e96ac4 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:42.808459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:42.907377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c941d5c-e4e4-4f26-a4ad-059492e96ac4 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:42.944420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427225a6-f69d-433b-92ea-7173455f0154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:42.945085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427225a6-f69d-433b-92ea-7173455f0154 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:42.945368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427225a6-f69d-433b-92ea-7173455f0154 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:42.945643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427225a6-f69d-433b-92ea-7173455f0154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:42.956682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427225a6-f69d-433b-92ea-7173455f0154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:42.958152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427225a6-f69d-433b-92ea-7173455f0154 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-427225a6-f69d-433b-92ea-7173455f0154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:42.972759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:42.973648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:42.974745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:42.974745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:42.974745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:42.974745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:42.974745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 466/923] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:42.981231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45941ed7-246a-481f-a00c-f1ce6a0d703e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:43.046434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:43.088371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45941ed7-246a-481f-a00c-f1ce6a0d703e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:43.124002 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9e8ed-1ad9-444a-a2eb-5d5276a02d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:43.124587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9e8ed-1ad9-444a-a2eb-5d5276a02d23 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:43.124865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9e8ed-1ad9-444a-a2eb-5d5276a02d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:43.125131 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9e8ed-1ad9-444a-a2eb-5d5276a02d23 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:43.125839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9e8ed-1ad9-444a-a2eb-5d5276a02d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:43.126146 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9e8ed-1ad9-444a-a2eb-5d5276a02d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:43.127618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c9e8ed-1ad9-444a-a2eb-5d5276a02d23 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:43.140590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 458/924] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:43.146024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427225a6-f69d-433b-92ea-7173455f0154 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:43.232233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:43.256985 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427225a6-f69d-433b-92ea-7173455f0154 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:43.293425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40708a-a483-4b98-bcf1-07536cee6426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:43.294076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40708a-a483-4b98-bcf1-07536cee6426 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:43.294350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40708a-a483-4b98-bcf1-07536cee6426 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:43.294617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40708a-a483-4b98-bcf1-07536cee6426 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:43.304004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40708a-a483-4b98-bcf1-07536cee6426 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:43.305499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40708a-a483-4b98-bcf1-07536cee6426 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-0d40708a-a483-4b98-bcf1-07536cee6426 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:43.317909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:43.318908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:43.319873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:43.319873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:43.319873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:43.319873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:43.319873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:43.319873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:43.319873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:43.319873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:43.319873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:43.319873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:43.323549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 467/925] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Aug 1 11:51:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:43.325389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9e8ed-1ad9-444a-a2eb-5d5276a02d23 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:43.430707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c9e8ed-1ad9-444a-a2eb-5d5276a02d23 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:43.458287 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:43.471523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94cef8-9031-45d6-863a-196260a136d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:43.472158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94cef8-9031-45d6-863a-196260a136d6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:43.472463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94cef8-9031-45d6-863a-196260a136d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:43.472733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94cef8-9031-45d6-863a-196260a136d6 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:43.473600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94cef8-9031-45d6-863a-196260a136d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:43.473907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94cef8-9031-45d6-863a-196260a136d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:43.475343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94cef8-9031-45d6-863a-196260a136d6 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:43.498528 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 459/926] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Aug 1 11:51:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:43.501111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40708a-a483-4b98-bcf1-07536cee6426 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:43.607772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40708a-a483-4b98-bcf1-07536cee6426 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:43.643081 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d4ede-f31e-478f-a85d-b1cefc3909fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:43.643736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d4ede-f31e-478f-a85d-b1cefc3909fb None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:43.644011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d4ede-f31e-478f-a85d-b1cefc3909fb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:43.644279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d4ede-f31e-478f-a85d-b1cefc3909fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:43.647329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:43.657597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d4ede-f31e-478f-a85d-b1cefc3909fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:43.659190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d4ede-f31e-478f-a85d-b1cefc3909fb None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-0b2d4ede-f31e-478f-a85d-b1cefc3909fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:43.675246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:43.676185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:43.677099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:43.677099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:43.677099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:43.677099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:43.680403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 468/927] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:43.682814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94cef8-9031-45d6-863a-196260a136d6 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:43.725765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae050a0-c64c-487b-a8dc-e79f3c175896 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:43.726104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae050a0-c64c-487b-a8dc-e79f3c175896 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:43.726382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae050a0-c64c-487b-a8dc-e79f3c175896 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:43.728142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 469/928] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:51:43.734480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc5394-4488-41e9-a967-8ecb48b8a8ef None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:43.734789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc5394-4488-41e9-a967-8ecb48b8a8ef None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:43.735008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc5394-4488-41e9-a967-8ecb48b8a8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:43.755291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-decc5394-4488-41e9-a967-8ecb48b8a8ef None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:43.756589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-decc5394-4488-41e9-a967-8ecb48b8a8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:43.796386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94cef8-9031-45d6-863a-196260a136d6 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:43.834105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4375fc2-5ce1-4743-ba42-9bec4aa477b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:43.834749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4375fc2-5ce1-4743-ba42-9bec4aa477b7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:43.835304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4375fc2-5ce1-4743-ba42-9bec4aa477b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:43.835581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4375fc2-5ce1-4743-ba42-9bec4aa477b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:43.836982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4375fc2-5ce1-4743-ba42-9bec4aa477b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:43.837422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4375fc2-5ce1-4743-ba42-9bec4aa477b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:43.838834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4375fc2-5ce1-4743-ba42-9bec4aa477b7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:43.841104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 470/929] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:43.851401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 460/930] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:43.859238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4375fc2-5ce1-4743-ba42-9bec4aa477b7 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:43.875480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc5394-4488-41e9-a967-8ecb48b8a8ef None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:43.925533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:44.003820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc5394-4488-41e9-a967-8ecb48b8a8ef None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:44.008042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4375fc2-5ce1-4743-ba42-9bec4aa477b7 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:44.024918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:44.045444 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacdf2e-f482-4d6d-9781-ea6ec3310047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:44.046092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacdf2e-f482-4d6d-9781-ea6ec3310047 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.046368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacdf2e-f482-4d6d-9781-ea6ec3310047 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.046641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacdf2e-f482-4d6d-9781-ea6ec3310047 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.047430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacdf2e-f482-4d6d-9781-ea6ec3310047 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:44.049474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacdf2e-f482-4d6d-9781-ea6ec3310047 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:44.056884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4136b0-0933-4e4a-9b9a-f2075c46193e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:44.057804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4136b0-0933-4e4a-9b9a-f2075c46193e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.058281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4136b0-0933-4e4a-9b9a-f2075c46193e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.058768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4136b0-0933-4e4a-9b9a-f2075c46193e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-ecacdf2e-f482-4d6d-9781-ea6ec3310047 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:44.059516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:44.060436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 01 11:51:44.061291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:44.064375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 471/931] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Aug 1 11:51:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:44.067600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacdf2e-f482-4d6d-9781-ea6ec3310047 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:44.073309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4136b0-0933-4e4a-9b9a-f2075c46193e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:44.075076 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4136b0-0933-4e4a-9b9a-f2075c46193e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-8b4136b0-0933-4e4a-9b9a-f2075c46193e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:44.091662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:44.092723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:44.093604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:44.093604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:44.093604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:44.093604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:44.093604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:44.093604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:44.093604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:44.093604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:44.097099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 461/932] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:44.102910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4136b0-0933-4e4a-9b9a-f2075c46193e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:44.179872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacdf2e-f482-4d6d-9781-ea6ec3310047 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:44.209298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:44.212444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4136b0-0933-4e4a-9b9a-f2075c46193e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:44.217201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70671351-f2db-4a8f-b358-786e37fde7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:44.218018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70671351-f2db-4a8f-b358-786e37fde7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.218472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70671351-f2db-4a8f-b358-786e37fde7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.218897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70671351-f2db-4a8f-b358-786e37fde7d4 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.219724 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70671351-f2db-4a8f-b358-786e37fde7d4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:44.220624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70671351-f2db-4a8f-b358-786e37fde7d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:44.222701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70671351-f2db-4a8f-b358-786e37fde7d4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:44.229831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 472/933] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:44.231258 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eb214-fd90-42f7-8cb0-18f142260fa9 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.231541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eb214-fd90-42f7-8cb0-18f142260fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.231812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36eb214-fd90-42f7-8cb0-18f142260fa9 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.233205 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 473/934] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:44.238330 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eb214-fd90-42f7-8cb0-18f142260fa9 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:44.257141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e15bc0-7f31-43d2-b96f-33ba4dacf1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:44.258092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15bc0-7f31-43d2-b96f-33ba4dacf1bc None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.258383 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15bc0-7f31-43d2-b96f-33ba4dacf1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.258738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e15bc0-7f31-43d2-b96f-33ba4dacf1bc None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.259572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e15bc0-7f31-43d2-b96f-33ba4dacf1bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:44.259909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e15bc0-7f31-43d2-b96f-33ba4dacf1bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:44.261448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e15bc0-7f31-43d2-b96f-33ba4dacf1bc None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:44.275020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 462/935] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:44.276311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8bfa7-838e-4d80-9c91-273827b161dd None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.276600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8bfa7-838e-4d80-9c91-273827b161dd None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.276873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8bfa7-838e-4d80-9c91-273827b161dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.294377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-2cb8bfa7-838e-4d80-9c91-273827b161dd None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:44.294907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:44.295723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2cb8bfa7-838e-4d80-9c91-273827b161dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:44.356242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36eb214-fd90-42f7-8cb0-18f142260fa9 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:44.363829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 463/936] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:44.364711 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8bfa7-838e-4d80-9c91-273827b161dd None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:44.400533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a9373-5f5f-42f4-ac30-713e3dc96443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:44.401273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a9373-5f5f-42f4-ac30-713e3dc96443 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.401596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a9373-5f5f-42f4-ac30-713e3dc96443 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.401916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a9373-5f5f-42f4-ac30-713e3dc96443 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.402667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a9373-5f5f-42f4-ac30-713e3dc96443 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:44.402974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a9373-5f5f-42f4-ac30-713e3dc96443 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:44.404402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a9373-5f5f-42f4-ac30-713e3dc96443 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:44.409763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 474/937] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:44.410565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a9373-5f5f-42f4-ac30-713e3dc96443 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:44.474884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:44.485777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8bfa7-838e-4d80-9c91-273827b161dd None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:44.518915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a9373-5f5f-42f4-ac30-713e3dc96443 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:44.520416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841df18-f8c2-4882-b074-8dbd13205b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:44.521112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841df18-f8c2-4882-b074-8dbd13205b2b None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.521392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841df18-f8c2-4882-b074-8dbd13205b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.521665 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841df18-f8c2-4882-b074-8dbd13205b2b None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/bf76a8ec03e444e7b42e2997b8f73808/roles/9a4d5a7314884a599e1b22288a1add9d` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.523877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841df18-f8c2-4882-b074-8dbd13205b2b None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=bf76a8ec03e444e7b42e2997b8f73808, role_id=9a4d5a7314884a599e1b22288a1add9d)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:44.525380 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841df18-f8c2-4882-b074-8dbd13205b2b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:44.530003 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:44.548419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 464/938] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:51:44 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/bf76a8ec03e444e7b42e2997b8f73808/roles/9a4d5a7314884a599e1b22288a1add9d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:51:44.549562 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841df18-f8c2-4882-b074-8dbd13205b2b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:44.558637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d631dfc-e9cd-40be-8e98-dc43285460db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:44.559318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d631dfc-e9cd-40be-8e98-dc43285460db None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.559613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d631dfc-e9cd-40be-8e98-dc43285460db None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.559876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d631dfc-e9cd-40be-8e98-dc43285460db None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.560638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d631dfc-e9cd-40be-8e98-dc43285460db None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:44.562158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d631dfc-e9cd-40be-8e98-dc43285460db None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-0d631dfc-e9cd-40be-8e98-dc43285460db None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:44.574141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:44.575244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:44.576099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 475/939] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:44.582879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d631dfc-e9cd-40be-8e98-dc43285460db None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:44.697763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841df18-f8c2-4882-b074-8dbd13205b2b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:44.717675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d631dfc-e9cd-40be-8e98-dc43285460db None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:44.744121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac89d42-b89c-4c05-8d4e-36d13a70fabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:44.744929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac89d42-b89c-4c05-8d4e-36d13a70fabd None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.745204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac89d42-b89c-4c05-8d4e-36d13a70fabd None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.745722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac89d42-b89c-4c05-8d4e-36d13a70fabd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.746205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac89d42-b89c-4c05-8d4e-36d13a70fabd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:44.747601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac89d42-b89c-4c05-8d4e-36d13a70fabd None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:44.758810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abb61f-4718-4ae1-a435-a72354d79442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:44.759429 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abb61f-4718-4ae1-a435-a72354d79442 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.760086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abb61f-4718-4ae1-a435-a72354d79442 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.760401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abb61f-4718-4ae1-a435-a72354d79442 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.761258 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abb61f-4718-4ae1-a435-a72354d79442 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:44.762004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abb61f-4718-4ae1-a435-a72354d79442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:44.763052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abb61f-4718-4ae1-a435-a72354d79442 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:44.774222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 476/940] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Aug 1 11:51:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:44.775689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abb61f-4718-4ae1-a435-a72354d79442 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:44.780322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 465/941] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 11:51:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:44.800817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061e1ab-11b5-4b61-97fc-f18be06a2fc4 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.801447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061e1ab-11b5-4b61-97fc-f18be06a2fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.801697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061e1ab-11b5-4b61-97fc-f18be06a2fc4 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.803911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 466/942] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:51:44.809928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba749-ed4b-4e07-8653-6291b546b688 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.810217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba749-ed4b-4e07-8653-6291b546b688 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.810556 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ba749-ed4b-4e07-8653-6291b546b688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.827002 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:44.831794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:44.839236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ef4ba749-ed4b-4e07-8653-6291b546b688 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:44.841204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ef4ba749-ed4b-4e07-8653-6291b546b688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:44.915323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abb61f-4718-4ae1-a435-a72354d79442 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:44.953750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 467/943] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:44.963830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b341d49-0323-49ff-8c92-2eb229100fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:44.964517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b341d49-0323-49ff-8c92-2eb229100fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:44.964798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b341d49-0323-49ff-8c92-2eb229100fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:44.965078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b341d49-0323-49ff-8c92-2eb229100fd2 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:44.965831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b341d49-0323-49ff-8c92-2eb229100fd2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:44.966132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b341d49-0323-49ff-8c92-2eb229100fd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:44.967570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b341d49-0323-49ff-8c92-2eb229100fd2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:44.974064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 477/944] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Aug 1 11:51:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:44.977270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba749-ed4b-4e07-8653-6291b546b688 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:44.979737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b341d49-0323-49ff-8c92-2eb229100fd2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:45.095120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ba749-ed4b-4e07-8653-6291b546b688 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:45.103386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b341d49-0323-49ff-8c92-2eb229100fd2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:45.105762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:45.131414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:45.138574 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:45.139535 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:45.140050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:45.140502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:45.141353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:45.143285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:45.146399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: WARNING keystone.server.flask.common [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 11:51:45.151586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fab3a-c9f9-4fc2-8168-ace90893f38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:45.152218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fab3a-c9f9-4fc2-8168-ace90893f38a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:45.152520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fab3a-c9f9-4fc2-8168-ace90893f38a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:45.152795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fab3a-c9f9-4fc2-8168-ace90893f38a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:45.153563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fab3a-c9f9-4fc2-8168-ace90893f38a None admin] RBAC: Adding query filter params (interface=public, service_id=dd3cca32a05043d4b4ca0dea1260647b, region_id=RegionOne) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:45.153870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fab3a-c9f9-4fc2-8168-ace90893f38a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:45.155298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9fab3a-c9f9-4fc2-8168-ace90893f38a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:45.162305 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 478/945] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Aug 1 11:51:44 2022] GET /identity/v3/endpoints?service_id=dd3cca32a05043d4b4ca0dea1260647b&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:45.167715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fab3a-c9f9-4fc2-8168-ace90893f38a None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:45.171918 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.sql.core [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 11:51:45.171918 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 11:51:45.171918 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [parameters: {'id': '2e7194f631f540e4a5494cacdf1316db', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 11:51:45.171918 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80163) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2e7194f631f540e4a5494cacdf1316db.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2e7194f631f540e4a5494cacdf1316db. Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application conn.query(q) Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 11:51:45.179690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application result.read() Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 01 11:51:45.180588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application next(self.gen) Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application yield resource Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application next(self.gen) Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise self.value Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application session.commit() Aug 01 11:51:45.181767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self.session.flush() Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application compat.raise_( Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 11:51:45.182960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 11:51:45.184205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application conn.query(q) Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application result.read() Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 11:51:45.185042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [parameters: {'id': '2e7194f631f540e4a5494cacdf1316db', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:45.186225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:45.187086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2e7194f631f540e4a5494cacdf1316db. Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 468/946] 10.222.0.49 () {60 vars in 1140 bytes} [Mon Aug 1 11:51:44 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:45.188099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:45.312342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9fab3a-c9f9-4fc2-8168-ace90893f38a None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:45.313636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02432986-34d6-445c-918f-37c167f627b8 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:45.350182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc808-47e8-4d86-98b2-e5047c80342b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:45.350912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc808-47e8-4d86-98b2-e5047c80342b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:45.351192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc808-47e8-4d86-98b2-e5047c80342b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:45.351463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbc808-47e8-4d86-98b2-e5047c80342b None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:45.352065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0ddbc-463d-4e14-8978-ab753ff08cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:45.352207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc808-47e8-4d86-98b2-e5047c80342b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:45.352859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0ddbc-463d-4e14-8978-ab753ff08cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:45.353205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0ddbc-463d-4e14-8978-ab753ff08cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:45.353545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0ddbc-463d-4e14-8978-ab753ff08cf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:45.353660 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbc808-47e8-4d86-98b2-e5047c80342b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:45.360511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 479/947] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Aug 1 11:51:45 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 11:51:45.369875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0ddbc-463d-4e14-8978-ab753ff08cf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:45.371672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0ddbc-463d-4e14-8978-ab753ff08cf3 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-4ea0ddbc-463d-4e14-8978-ab753ff08cf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:45.385074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:45.386065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:45.387006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:45.387006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:45.387006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:45.387006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:45.387006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:45.387006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:45.387006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:45.387006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:45.387006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 469/948] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:45.392835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc808-47e8-4d86-98b2-e5047c80342b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:45.432716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:45.503412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbc808-47e8-4d86-98b2-e5047c80342b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:45.540815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eadbcf-e39e-4968-8bdb-cb6c6ac49215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:45.541448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eadbcf-e39e-4968-8bdb-cb6c6ac49215 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:45.541722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eadbcf-e39e-4968-8bdb-cb6c6ac49215 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:45.541988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eadbcf-e39e-4968-8bdb-cb6c6ac49215 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:45.542713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eadbcf-e39e-4968-8bdb-cb6c6ac49215 None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:45.543012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eadbcf-e39e-4968-8bdb-cb6c6ac49215 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:45.544479 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eadbcf-e39e-4968-8bdb-cb6c6ac49215 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:45.562560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 480/949] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:46.651057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec3900-c8c4-49df-beff-187f1ce6df6b None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:46.651800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec3900-c8c4-49df-beff-187f1ce6df6b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:46.651800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec3900-c8c4-49df-beff-187f1ce6df6b None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:46.656036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 470/950] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:46.661684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271936-5f5a-4bee-b1f0-fcde22b4daca None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:46.662553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271936-5f5a-4bee-b1f0-fcde22b4daca None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:46.662553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271936-5f5a-4bee-b1f0-fcde22b4daca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:46.686035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-75271936-5f5a-4bee-b1f0-fcde22b4daca None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:46.687268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-75271936-5f5a-4bee-b1f0-fcde22b4daca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:46.765075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 481/951] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:46.787293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec3900-c8c4-49df-beff-187f1ce6df6b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:46.843048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:46.903842 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec3900-c8c4-49df-beff-187f1ce6df6b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:46.921788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df16778-e909-45b3-b75b-2a22d0afc044 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:46.922421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df16778-e909-45b3-b75b-2a22d0afc044 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:46.922919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df16778-e909-45b3-b75b-2a22d0afc044 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:46.924820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 482/952] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:46.929946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9a076-bd70-443d-ae38-48b2150e1790 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:46.930476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9a076-bd70-443d-ae38-48b2150e1790 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:46.930949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9a076-bd70-443d-ae38-48b2150e1790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:46.939108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828b65b-6bb1-4fb7-a0d0-1b86306953da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:46.939848 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b65b-6bb1-4fb7-a0d0-1b86306953da None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:46.940132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b65b-6bb1-4fb7-a0d0-1b86306953da None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:46.940441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828b65b-6bb1-4fb7-a0d0-1b86306953da None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:46.951304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9ca9a076-bd70-443d-ae38-48b2150e1790 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:46.951696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828b65b-6bb1-4fb7-a0d0-1b86306953da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:46.953168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9ca9a076-bd70-443d-ae38-48b2150e1790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-7828b65b-6bb1-4fb7-a0d0-1b86306953da None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:46.956972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:46.957939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:46.958769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 471/953] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Aug 1 11:51:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:46.964563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828b65b-6bb1-4fb7-a0d0-1b86306953da None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:46.979745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:47.038509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 483/954] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:47.060811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9a076-bd70-443d-ae38-48b2150e1790 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:47.095415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828b65b-6bb1-4fb7-a0d0-1b86306953da None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:47.106211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:47.134427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8b276-e377-40e4-9e1d-37750ed019be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:47.135232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8b276-e377-40e4-9e1d-37750ed019be None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.135716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8b276-e377-40e4-9e1d-37750ed019be None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.136089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8b276-e377-40e4-9e1d-37750ed019be None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.136965 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8b276-e377-40e4-9e1d-37750ed019be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:47.137373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8b276-e377-40e4-9e1d-37750ed019be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:47.139140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a8b276-e377-40e4-9e1d-37750ed019be None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:47.153698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 472/955] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Aug 1 11:51:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:47.154977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8b276-e377-40e4-9e1d-37750ed019be None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:47.194094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9a076-bd70-443d-ae38-48b2150e1790 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:47.240740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ba78c-133d-4fe1-9e7d-61c84dcbebf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:47.241401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ba78c-133d-4fe1-9e7d-61c84dcbebf3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.241679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ba78c-133d-4fe1-9e7d-61c84dcbebf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.241955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ba78c-133d-4fe1-9e7d-61c84dcbebf3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.242595 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ba78c-133d-4fe1-9e7d-61c84dcbebf3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:47.244563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ba78c-133d-4fe1-9e7d-61c84dcbebf3 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-5a3ba78c-133d-4fe1-9e7d-61c84dcbebf3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:47.256728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:47.258141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:47.259284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 484/956] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:47.264955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ba78c-133d-4fe1-9e7d-61c84dcbebf3 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:47.274684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8b276-e377-40e4-9e1d-37750ed019be None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:47.285886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:47.314872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2416a-f127-498c-b322-be778325a075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:47.315738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2416a-f127-498c-b322-be778325a075 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.316116 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2416a-f127-498c-b322-be778325a075 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.316528 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2416a-f127-498c-b322-be778325a075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.329083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2416a-f127-498c-b322-be778325a075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:47.330901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2416a-f127-498c-b322-be778325a075 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-51f2416a-f127-498c-b322-be778325a075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:47.347055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:47.348189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:47.350000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:47.350000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:47.350000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:47.350000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:47.354604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 473/957] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:47.355595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2416a-f127-498c-b322-be778325a075 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:47.385887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:47.397381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ba78c-133d-4fe1-9e7d-61c84dcbebf3 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:47.445008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e0f32-31e8-44c1-a0e2-fac86db2312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:47.445880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e0f32-31e8-44c1-a0e2-fac86db2312f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.446323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e0f32-31e8-44c1-a0e2-fac86db2312f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.447312 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e0f32-31e8-44c1-a0e2-fac86db2312f None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.448157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e0f32-31e8-44c1-a0e2-fac86db2312f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:47.448629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e0f32-31e8-44c1-a0e2-fac86db2312f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:47.450975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e0f32-31e8-44c1-a0e2-fac86db2312f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:47.458538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 485/958] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:47.460144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ea381-6f6e-42f8-b94e-849211850bed None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.460614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ea381-6f6e-42f8-b94e-849211850bed None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.461037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ea381-6f6e-42f8-b94e-849211850bed None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.463510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 486/959] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:47.466133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ea381-6f6e-42f8-b94e-849211850bed None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:47.474345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2416a-f127-498c-b322-be778325a075 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:47.514463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cdb4f-e56a-4198-b162-a2683b11c303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:47.515149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cdb4f-e56a-4198-b162-a2683b11c303 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.515432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cdb4f-e56a-4198-b162-a2683b11c303 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.515703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cdb4f-e56a-4198-b162-a2683b11c303 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.516584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cdb4f-e56a-4198-b162-a2683b11c303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:47.516896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cdb4f-e56a-4198-b162-a2683b11c303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:47.518401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cdb4f-e56a-4198-b162-a2683b11c303 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:47.532366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 474/960] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:47.533768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9d721-533b-4b20-a1b3-089d520250c8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.534054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9d721-533b-4b20-a1b3-089d520250c8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.534331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9d721-533b-4b20-a1b3-089d520250c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.553852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-eca9d721-533b-4b20-a1b3-089d520250c8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:47.555561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-eca9d721-533b-4b20-a1b3-089d520250c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:47.556803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:47.581379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ea381-6f6e-42f8-b94e-849211850bed None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:47.616842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094660e-751f-4e1f-a19f-0fb9efec6301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:47.617478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094660e-751f-4e1f-a19f-0fb9efec6301 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.617750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094660e-751f-4e1f-a19f-0fb9efec6301 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.618455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094660e-751f-4e1f-a19f-0fb9efec6301 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.619149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094660e-751f-4e1f-a19f-0fb9efec6301 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:47.619748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094660e-751f-4e1f-a19f-0fb9efec6301 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:47.621024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e094660e-751f-4e1f-a19f-0fb9efec6301 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:47.625909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 487/961] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:47.626571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094660e-751f-4e1f-a19f-0fb9efec6301 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:47.635061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 475/962] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:47.637570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec3762-38eb-445b-809c-3a6433abb953 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.638169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec3762-38eb-445b-809c-3a6433abb953 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.638657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec3762-38eb-445b-809c-3a6433abb953 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.641444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 476/963] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:47.646473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb673be-2dc7-4270-8156-114a4a17bd05 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.646869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb673be-2dc7-4270-8156-114a4a17bd05 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.647080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb673be-2dc7-4270-8156-114a4a17bd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.667432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-9eb673be-2dc7-4270-8156-114a4a17bd05 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:47.668704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9eb673be-2dc7-4270-8156-114a4a17bd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:47.677744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:47.746783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 477/964] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:47.755459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094660e-751f-4e1f-a19f-0fb9efec6301 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:47.757490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb673be-2dc7-4270-8156-114a4a17bd05 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:47.798158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83324e91-b2fa-4833-810c-167448928d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:47.798858 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83324e91-b2fa-4833-810c-167448928d1d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.799310 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83324e91-b2fa-4833-810c-167448928d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.799591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83324e91-b2fa-4833-810c-167448928d1d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.811009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83324e91-b2fa-4833-810c-167448928d1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:47.812873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83324e91-b2fa-4833-810c-167448928d1d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-83324e91-b2fa-4833-810c-167448928d1d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:47.825188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:47.826197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:47.827145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 488/965] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Aug 1 11:51:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:47.828243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83324e91-b2fa-4833-810c-167448928d1d None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:47.835117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:47.879180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb673be-2dc7-4270-8156-114a4a17bd05 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:47.915713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a68dbe-dd03-4c20-a5db-cdef0ef654db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:47.916405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a68dbe-dd03-4c20-a5db-cdef0ef654db None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.916689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a68dbe-dd03-4c20-a5db-cdef0ef654db None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.916962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a68dbe-dd03-4c20-a5db-cdef0ef654db None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.917598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a68dbe-dd03-4c20-a5db-cdef0ef654db None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:47.919480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a68dbe-dd03-4c20-a5db-cdef0ef654db None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-38a68dbe-dd03-4c20-a5db-cdef0ef654db None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:47.929518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:47.930564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:47.931467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 478/966] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:47.932443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a68dbe-dd03-4c20-a5db-cdef0ef654db None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:47.943600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:47.949768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83324e91-b2fa-4833-810c-167448928d1d None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:47.987184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:47.987826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:47.988103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:47.988420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:47.989109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:47.990699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:47.993571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WARNING keystone.server.flask.common [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 01 11:51:48.007423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.sql.core [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 11:51:48.007423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 11:51:48.007423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [parameters: {'user_id': '38c9c96ecf764db58ca74ad380f3e0de', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 11:51:48.007423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80164) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application conn.query(q) Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application result.read() Aug 01 11:51:48.025607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 11:51:48.026638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application next(self.gen) Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application yield resource Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application next(self.gen) Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self.session.rollback() Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self.force_reraise() Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise self.value Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application session.commit() Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 01 11:51:48.027728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._prepare_impl() Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self.session.flush() Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._flush(objects) Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application compat.raise_( Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application flush_context.execute() Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rec.execute(self) Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 01 11:51:48.028806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 01 11:51:48.029961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application result = self._query(query) Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application conn.query(q) Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application result.read() Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 01 11:51:48.030941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [parameters: {'user_id': '38c9c96ecf764db58ca74ad380f3e0de', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:48.031942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 01 11:51:48.033455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 01 11:51:48.035536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 01 11:51:48.035536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 01 11:51:48.035536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 01 11:51:48.035536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 01 11:51:48.035536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 01 11:51:48.035536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:48.035536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 489/967] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Aug 1 11:51:47 2022] POST /identity/v3/users => generated 160 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:48.035536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:48.047048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a68dbe-dd03-4c20-a5db-cdef0ef654db None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:48.090329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd78bb-f263-4e38-9239-1739a15ad092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:48.090976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd78bb-f263-4e38-9239-1739a15ad092 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:48.091259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd78bb-f263-4e38-9239-1739a15ad092 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:48.091537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd78bb-f263-4e38-9239-1739a15ad092 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:48.092378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd78bb-f263-4e38-9239-1739a15ad092 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:48.092693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd78bb-f263-4e38-9239-1739a15ad092 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:48.094136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd78bb-f263-4e38-9239-1739a15ad092 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:48.096572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:48.118020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 479/968] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Aug 1 11:51:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:48.118922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd78bb-f263-4e38-9239-1739a15ad092 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:48.144717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea0cdc-c477-42da-a645-4280eb14e947 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:48.182290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebdebf-89c0-408a-9236-ba93e7970aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:48.182926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebdebf-89c0-408a-9236-ba93e7970aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:48.183216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebdebf-89c0-408a-9236-ba93e7970aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:48.183490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebdebf-89c0-408a-9236-ba93e7970aa7 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:48.184178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebdebf-89c0-408a-9236-ba93e7970aa7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:48.184513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebdebf-89c0-408a-9236-ba93e7970aa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:48.186116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ebdebf-89c0-408a-9236-ba93e7970aa7 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:48.194381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:48.195111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 490/969] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Aug 1 11:51:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:48.196053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebdebf-89c0-408a-9236-ba93e7970aa7 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:48.245476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd78bb-f263-4e38-9239-1739a15ad092 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:48.281446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c9ec6-9239-40b3-a1e2-b265d669838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:48.282016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c9ec6-9239-40b3-a1e2-b265d669838a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:48.282295 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c9ec6-9239-40b3-a1e2-b265d669838a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:48.282571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c9ec6-9239-40b3-a1e2-b265d669838a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:48.290243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c9ec6-9239-40b3-a1e2-b265d669838a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:48.291695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c9ec6-9239-40b3-a1e2-b265d669838a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-f23c9ec6-9239-40b3-a1e2-b265d669838a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:48.303738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:48.304796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 480/970] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:51:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:48.305769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c9ec6-9239-40b3-a1e2-b265d669838a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:48.306921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebdebf-89c0-408a-9236-ba93e7970aa7 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:48.336378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:48.345883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c48869-a567-4345-a4d9-530f93589a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:48.346750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48869-a567-4345-a4d9-530f93589a50 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:48.347183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48869-a567-4345-a4d9-530f93589a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:48.347609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c48869-a567-4345-a4d9-530f93589a50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:48.412603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c9ec6-9239-40b3-a1e2-b265d669838a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:48.416848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c48869-a567-4345-a4d9-530f93589a50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:48.418328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c48869-a567-4345-a4d9-530f93589a50 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:48.441885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:48.455616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65490e-5037-4456-af7b-da77a413524c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:48.456251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65490e-5037-4456-af7b-da77a413524c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:48.456565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65490e-5037-4456-af7b-da77a413524c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:48.456829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65490e-5037-4456-af7b-da77a413524c None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:48.457585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65490e-5037-4456-af7b-da77a413524c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:48.457892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65490e-5037-4456-af7b-da77a413524c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:48.459282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a65490e-5037-4456-af7b-da77a413524c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:48.464273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 481/971] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Aug 1 11:51:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:48.465071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65490e-5037-4456-af7b-da77a413524c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:48.533437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 491/972] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:48 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:51:48.534186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c48869-a567-4345-a4d9-530f93589a50 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:48.565850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:48.578832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65490e-5037-4456-af7b-da77a413524c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:48.614083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e556833-5350-44c6-9695-3461679a3c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:48.614726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e556833-5350-44c6-9695-3461679a3c5c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:48.615008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e556833-5350-44c6-9695-3461679a3c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:48.615279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e556833-5350-44c6-9695-3461679a3c5c None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:48.616042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e556833-5350-44c6-9695-3461679a3c5c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:48.616372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e556833-5350-44c6-9695-3461679a3c5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:48.617827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e556833-5350-44c6-9695-3461679a3c5c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:48.647221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c48869-a567-4345-a4d9-530f93589a50 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:48.664330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 482/973] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Aug 1 11:51:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:48.664330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e556833-5350-44c6-9695-3461679a3c5c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:48.704220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b254a66-d25e-4b40-8a7c-0b21d0a01b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:48.705036 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b254a66-d25e-4b40-8a7c-0b21d0a01b5b None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:48.705390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b254a66-d25e-4b40-8a7c-0b21d0a01b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:48.705738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b254a66-d25e-4b40-8a7c-0b21d0a01b5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:48.706586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b254a66-d25e-4b40-8a7c-0b21d0a01b5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:48.708115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b254a66-d25e-4b40-8a7c-0b21d0a01b5b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:48.728328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:48.744341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 492/974] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 11:51:48 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:48.805184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e556833-5350-44c6-9695-3461679a3c5c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:48.858019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea65f7-dc38-4b04-8a42-79b2ce579203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:48.858019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea65f7-dc38-4b04-8a42-79b2ce579203 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:48.858019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea65f7-dc38-4b04-8a42-79b2ce579203 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:48.858019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea65f7-dc38-4b04-8a42-79b2ce579203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:48.869809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea65f7-dc38-4b04-8a42-79b2ce579203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:48.871309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea65f7-dc38-4b04-8a42-79b2ce579203 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-d2ea65f7-dc38-4b04-8a42-79b2ce579203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:48.888269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:48.889262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:48.890311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:48.890311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:48.890311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:48.890311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:48.890995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 483/975] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:48.896573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b254a66-d25e-4b40-8a7c-0b21d0a01b5b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:48.995972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:49.014699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b254a66-d25e-4b40-8a7c-0b21d0a01b5b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:49.049590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c2865-c8ec-41a9-a933-aadc7e5ad4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:49.050205 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c2865-c8ec-41a9-a933-aadc7e5ad4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:49.050499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c2865-c8ec-41a9-a933-aadc7e5ad4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:49.050770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c2865-c8ec-41a9-a933-aadc7e5ad4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:49.051473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c2865-c8ec-41a9-a933-aadc7e5ad4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:49.051774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c2865-c8ec-41a9-a933-aadc7e5ad4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:49.053182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c2865-c8ec-41a9-a933-aadc7e5ad4d3 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:49.065766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 493/976] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:49.071208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea65f7-dc38-4b04-8a42-79b2ce579203 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:49.172474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:49.180637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea65f7-dc38-4b04-8a42-79b2ce579203 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:49.214337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3c4a1-68a7-4248-a5e5-40bc79ecc769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:49.214991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3c4a1-68a7-4248-a5e5-40bc79ecc769 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:49.215269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3c4a1-68a7-4248-a5e5-40bc79ecc769 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:49.215541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3c4a1-68a7-4248-a5e5-40bc79ecc769 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:49.222730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3c4a1-68a7-4248-a5e5-40bc79ecc769 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:49.224190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c3c4a1-68a7-4248-a5e5-40bc79ecc769 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-e4c3c4a1-68a7-4248-a5e5-40bc79ecc769 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:49.236195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:49.237349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:49.238293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:49.238293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:49.238293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:49.238293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:49.238293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:49.238293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:49.238293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:49.238293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:49.240374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 484/977] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:49.243336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c2865-c8ec-41a9-a933-aadc7e5ad4d3 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:49.352568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c2865-c8ec-41a9-a933-aadc7e5ad4d3 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:49.389189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d50d6d-0913-471b-b3f6-8a1757c23b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:49.389818 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d50d6d-0913-471b-b3f6-8a1757c23b0c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:49.390092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d50d6d-0913-471b-b3f6-8a1757c23b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:49.390345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d50d6d-0913-471b-b3f6-8a1757c23b0c None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:49.391120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d50d6d-0913-471b-b3f6-8a1757c23b0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:49.391433 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d50d6d-0913-471b-b3f6-8a1757c23b0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:49.392976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d50d6d-0913-471b-b3f6-8a1757c23b0c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:49.410405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 494/978] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:49.412056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3c4a1-68a7-4248-a5e5-40bc79ecc769 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:49.515044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3c4a1-68a7-4248-a5e5-40bc79ecc769 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:49.550618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f89129-f215-4e6f-b900-bb08abdc8a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:49.551268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89129-f215-4e6f-b900-bb08abdc8a61 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:49.551553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89129-f215-4e6f-b900-bb08abdc8a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:49.551817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89129-f215-4e6f-b900-bb08abdc8a61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:49.552722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f89129-f215-4e6f-b900-bb08abdc8a61 None admin] RBAC: Adding query filter params (role.id=9a4d5a7314884a599e1b22288a1add9d, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=bf76a8ec03e444e7b42e2997b8f73808) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:49.553067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f89129-f215-4e6f-b900-bb08abdc8a61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:49.554560 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f89129-f215-4e6f-b900-bb08abdc8a61 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:49.568349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 485/979] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:51:49 2022] GET /identity/v3/role_assignments?user.id=bf76a8ec03e444e7b42e2997b8f73808&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=9a4d5a7314884a599e1b22288a1add9d => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:49.663887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e36d0f-da30-458c-8e58-3ed9a8ef264d None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:49.664088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e36d0f-da30-458c-8e58-3ed9a8ef264d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:49.664394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e36d0f-da30-458c-8e58-3ed9a8ef264d None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:49.665844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 495/980] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:49.671617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502b08d-a582-4fe5-a31a-b9aa051b2919 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:49.671962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502b08d-a582-4fe5-a31a-b9aa051b2919 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:49.672150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e502b08d-a582-4fe5-a31a-b9aa051b2919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:49.691670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e502b08d-a582-4fe5-a31a-b9aa051b2919 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:49.692935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e502b08d-a582-4fe5-a31a-b9aa051b2919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:49.708477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:49.769267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 486/981] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:49.791455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e36d0f-da30-458c-8e58-3ed9a8ef264d None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:49.795391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:49.908891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e36d0f-da30-458c-8e58-3ed9a8ef264d None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:49.944632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58631658-a83c-4c7c-8060-44291ab1624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:49.945183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58631658-a83c-4c7c-8060-44291ab1624b None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:49.945458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58631658-a83c-4c7c-8060-44291ab1624b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:49.945730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58631658-a83c-4c7c-8060-44291ab1624b None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:49.946436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58631658-a83c-4c7c-8060-44291ab1624b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:49.947892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58631658-a83c-4c7c-8060-44291ab1624b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:49.965454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 496/982] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:51:49 2022] POST /identity/v3/services => generated 242 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:50.801381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5f4fc-afa5-4d0d-82fa-0929de5c03e3 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:50.801947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5f4fc-afa5-4d0d-82fa-0929de5c03e3 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:50.801947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5f4fc-afa5-4d0d-82fa-0929de5c03e3 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:50.803384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 487/983] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:50.808826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16dc83-f418-462c-a4c9-8f2531f70148 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:50.809118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16dc83-f418-462c-a4c9-8f2531f70148 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:50.809371 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16dc83-f418-462c-a4c9-8f2531f70148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:50.830981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-db16dc83-f418-462c-a4c9-8f2531f70148 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:50.832580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-db16dc83-f418-462c-a4c9-8f2531f70148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:50.896002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:50.913946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 497/984] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:50.936159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5f4fc-afa5-4d0d-82fa-0929de5c03e3 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:51.041052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5f4fc-afa5-4d0d-82fa-0929de5c03e3 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:51.076846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db800e-c937-4c0c-88dd-8c17834e2887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:51.077439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db800e-c937-4c0c-88dd-8c17834e2887 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:51.077718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db800e-c937-4c0c-88dd-8c17834e2887 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:51.077989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db800e-c937-4c0c-88dd-8c17834e2887 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:51.078634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db800e-c937-4c0c-88dd-8c17834e2887 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:51.080158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db800e-c937-4c0c-88dd-8c17834e2887 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-d7db800e-c937-4c0c-88dd-8c17834e2887 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:51.093757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:51.094861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:51.095773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:51.100174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 488/985] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:51.101483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16dc83-f418-462c-a4c9-8f2531f70148 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:51.207747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16dc83-f418-462c-a4c9-8f2531f70148 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:51.242566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686a70a-fbfa-4b84-bec9-936aea2ac7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:51.243206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a70a-fbfa-4b84-bec9-936aea2ac7ff None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:51.243490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a70a-fbfa-4b84-bec9-936aea2ac7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:51.243765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686a70a-fbfa-4b84-bec9-936aea2ac7ff None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:51.244471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686a70a-fbfa-4b84-bec9-936aea2ac7ff None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:51.244747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686a70a-fbfa-4b84-bec9-936aea2ac7ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:51.246215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686a70a-fbfa-4b84-bec9-936aea2ac7ff None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:51.255961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 498/986] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Aug 1 11:51:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:51.257271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db800e-c937-4c0c-88dd-8c17834e2887 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:51.273954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:51.363995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db800e-c937-4c0c-88dd-8c17834e2887 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:51.397941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c04a4f-3585-4142-9a94-10281daa2948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:51.398861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04a4f-3585-4142-9a94-10281daa2948 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:51.399226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04a4f-3585-4142-9a94-10281daa2948 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:51.399319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04a4f-3585-4142-9a94-10281daa2948 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:51.400030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c04a4f-3585-4142-9a94-10281daa2948 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:51.400412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c04a4f-3585-4142-9a94-10281daa2948 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:51.401785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c04a4f-3585-4142-9a94-10281daa2948 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:51.407229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 489/987] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Aug 1 11:51:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:51.413177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686a70a-fbfa-4b84-bec9-936aea2ac7ff None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:51.473683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:51.521781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686a70a-fbfa-4b84-bec9-936aea2ac7ff None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:51.557568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e38239-2d9d-472e-9264-f49865057ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:51.558273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38239-2d9d-472e-9264-f49865057ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:51.558538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38239-2d9d-472e-9264-f49865057ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:51.558809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e38239-2d9d-472e-9264-f49865057ae5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:51.559582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e38239-2d9d-472e-9264-f49865057ae5 None admin] RBAC: Adding query filter params (interface=admin, service_id=dd3cca32a05043d4b4ca0dea1260647b, region_id=RegionOne) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:51.559895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e38239-2d9d-472e-9264-f49865057ae5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:51.561309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e38239-2d9d-472e-9264-f49865057ae5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:51.567152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 499/988] 10.222.0.49 () {56 vars in 1249 bytes} [Mon Aug 1 11:51:51 2022] GET /identity/v3/endpoints?service_id=dd3cca32a05043d4b4ca0dea1260647b&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:51.572879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c04a4f-3585-4142-9a94-10281daa2948 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:51.595603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cf207-d3e8-4ea7-83ec-4f6b584b8bd2 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:51.595916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cf207-d3e8-4ea7-83ec-4f6b584b8bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:51.596153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cf207-d3e8-4ea7-83ec-4f6b584b8bd2 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:51.597553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 500/989] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:51.602566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d0a05-c3ce-49c2-bc1f-96a5bf2e7364 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:51.602850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d0a05-c3ce-49c2-bc1f-96a5bf2e7364 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:51.603187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d0a05-c3ce-49c2-bc1f-96a5bf2e7364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:51.621933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-124d0a05-c3ce-49c2-bc1f-96a5bf2e7364 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:51.623048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-124d0a05-c3ce-49c2-bc1f-96a5bf2e7364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:51.652922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:51.677344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c04a4f-3585-4142-9a94-10281daa2948 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:51.695871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 501/990] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:51.711247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072ef2a-4c7f-4580-a8b2-4edd40c035f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:51.711877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072ef2a-4c7f-4580-a8b2-4edd40c035f2 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:51.712165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072ef2a-4c7f-4580-a8b2-4edd40c035f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:51.712478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072ef2a-4c7f-4580-a8b2-4edd40c035f2 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:51.713135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072ef2a-4c7f-4580-a8b2-4edd40c035f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:51.714506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072ef2a-4c7f-4580-a8b2-4edd40c035f2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:51.716932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d0a05-c3ce-49c2-bc1f-96a5bf2e7364 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:51.721350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 490/991] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Aug 1 11:51:51 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 11:51:51.774173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:51.839502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d0a05-c3ce-49c2-bc1f-96a5bf2e7364 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:51.880955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1549a7-372e-4a8e-9494-4d908907aa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:51.881541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1549a7-372e-4a8e-9494-4d908907aa85 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:51.881822 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1549a7-372e-4a8e-9494-4d908907aa85 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:51.882095 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1549a7-372e-4a8e-9494-4d908907aa85 None admin] PATH_INFO: `/v3/services/image` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:51.882735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1549a7-372e-4a8e-9494-4d908907aa85 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:51.884124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1549a7-372e-4a8e-9494-4d908907aa85 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-8f1549a7-372e-4a8e-9494-4d908907aa85 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:51.893919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:51.894966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:51.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 502/992] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Aug 1 11:51:51 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:51.902099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072ef2a-4c7f-4580-a8b2-4edd40c035f2 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:51.979683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeb637-ff9b-406a-a78d-7427682a697f None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:51.980224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeb637-ff9b-406a-a78d-7427682a697f None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:51.980224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeb637-ff9b-406a-a78d-7427682a697f None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:51.981779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 503/993] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:51.989849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb1534-abc9-4e7c-97a4-5b27c21011cd None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:51.990278 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb1534-abc9-4e7c-97a4-5b27c21011cd None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:51.990533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb1534-abc9-4e7c-97a4-5b27c21011cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:52.009311 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-bffb1534-abc9-4e7c-97a4-5b27c21011cd None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:52.010585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bffb1534-abc9-4e7c-97a4-5b27c21011cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:52.011513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072ef2a-4c7f-4580-a8b2-4edd40c035f2 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:52.047672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad069-6c3e-4dde-99d4-303b52350600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:52.048400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad069-6c3e-4dde-99d4-303b52350600 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:52.048683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad069-6c3e-4dde-99d4-303b52350600 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:52.048958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad069-6c3e-4dde-99d4-303b52350600 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:52.049648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad069-6c3e-4dde-99d4-303b52350600 None admin] RBAC: Adding query filter params (name=image) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:52.049953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad069-6c3e-4dde-99d4-303b52350600 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:52.051377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad069-6c3e-4dde-99d4-303b52350600 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:52.057244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 491/994] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:52.063030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad069-6c3e-4dde-99d4-303b52350600 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:52.067958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:52.087273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 504/995] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:52.109493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb1534-abc9-4e7c-97a4-5b27c21011cd None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:52.158463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:52.178129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad069-6c3e-4dde-99d4-303b52350600 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:52.220787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c67cf4-68b5-4b7f-ad0a-163bf326997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:52.221417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67cf4-68b5-4b7f-ad0a-163bf326997b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:52.221701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67cf4-68b5-4b7f-ad0a-163bf326997b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:52.221974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c67cf4-68b5-4b7f-ad0a-163bf326997b None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:52.222638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb1534-abc9-4e7c-97a4-5b27c21011cd None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:52.222747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c67cf4-68b5-4b7f-ad0a-163bf326997b None admin] RBAC: Adding query filter params (type=image) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:52.222949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c67cf4-68b5-4b7f-ad0a-163bf326997b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:52.224871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c67cf4-68b5-4b7f-ad0a-163bf326997b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:52.231228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 492/996] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:52 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:52.276530 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd073b8a-8f4e-4ad6-8bcb-d565d98a889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:52.277194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd073b8a-8f4e-4ad6-8bcb-d565d98a889d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:52.277477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd073b8a-8f4e-4ad6-8bcb-d565d98a889d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:52.277739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd073b8a-8f4e-4ad6-8bcb-d565d98a889d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:52.278386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd073b8a-8f4e-4ad6-8bcb-d565d98a889d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:52.279824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd073b8a-8f4e-4ad6-8bcb-d565d98a889d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-dd073b8a-8f4e-4ad6-8bcb-d565d98a889d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:52.290309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:52.291511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:52.292385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 505/997] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:52.297880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c67cf4-68b5-4b7f-ad0a-163bf326997b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:52.373220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:52.413039 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c67cf4-68b5-4b7f-ad0a-163bf326997b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:52.449485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc048-871e-468a-8c35-a0fd87d2ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:52.450169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc048-871e-468a-8c35-a0fd87d2ed8f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:52.450447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc048-871e-468a-8c35-a0fd87d2ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:52.450716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1cc048-871e-468a-8c35-a0fd87d2ed8f None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:52.451399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cc048-871e-468a-8c35-a0fd87d2ed8f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:52.451701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cc048-871e-468a-8c35-a0fd87d2ed8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:52.453110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1cc048-871e-468a-8c35-a0fd87d2ed8f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:52.458785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 493/998] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:52.464572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd073b8a-8f4e-4ad6-8bcb-d565d98a889d None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:52.572449 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd073b8a-8f4e-4ad6-8bcb-d565d98a889d None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:52.581008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:52.610582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc0592-0665-4fb3-a70d-0a1feffe64b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:52.611590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc0592-0665-4fb3-a70d-0a1feffe64b5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:52.612060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc0592-0665-4fb3-a70d-0a1feffe64b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:52.612535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc0592-0665-4fb3-a70d-0a1feffe64b5 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:52.613394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc0592-0665-4fb3-a70d-0a1feffe64b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:52.613881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc0592-0665-4fb3-a70d-0a1feffe64b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:52.615444 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc0592-0665-4fb3-a70d-0a1feffe64b5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:52.620617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 506/999] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:52.626842 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc048-871e-468a-8c35-a0fd87d2ed8f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:52.732970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1cc048-871e-468a-8c35-a0fd87d2ed8f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:52.764327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:52.770160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50b46f-1ecb-448d-bf27-ca01f62fbef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:52.770796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50b46f-1ecb-448d-bf27-ca01f62fbef9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:52.771080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50b46f-1ecb-448d-bf27-ca01f62fbef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:52.771354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50b46f-1ecb-448d-bf27-ca01f62fbef9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:52.772120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50b46f-1ecb-448d-bf27-ca01f62fbef9 None admin] RBAC: Adding query filter params (interface=public, service_id=5da696bda3cd42ddae31c39edfe815d6, region_id=RegionOne) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:52.772641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50b46f-1ecb-448d-bf27-ca01f62fbef9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:52.774335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a50b46f-1ecb-448d-bf27-ca01f62fbef9 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:52.779975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 494/1000] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Aug 1 11:51:52 2022] GET /identity/v3/endpoints?service_id=5da696bda3cd42ddae31c39edfe815d6&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:52.785635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc0592-0665-4fb3-a70d-0a1feffe64b5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:52.897601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc0592-0665-4fb3-a70d-0a1feffe64b5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:52.933443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04378d85-3fa1-440c-add7-ff91148aca64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:52.934583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04378d85-3fa1-440c-add7-ff91148aca64 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:52.935154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04378d85-3fa1-440c-add7-ff91148aca64 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:52.935659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04378d85-3fa1-440c-add7-ff91148aca64 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:52.936555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04378d85-3fa1-440c-add7-ff91148aca64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:52.938155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04378d85-3fa1-440c-add7-ff91148aca64 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:52.945382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 507/1001] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Aug 1 11:51:52 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 11:51:53.726051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25266e63-2fa7-4f9e-8f4c-c2f595a5883f None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:53.726602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25266e63-2fa7-4f9e-8f4c-c2f595a5883f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:53.726602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25266e63-2fa7-4f9e-8f4c-c2f595a5883f None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:53.728110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 495/1002] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:51:53.733716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b39396-b775-4275-b6b0-1415834aae67 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:53.734145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b39396-b775-4275-b6b0-1415834aae67 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:53.734239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b39396-b775-4275-b6b0-1415834aae67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:53.752765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c2b39396-b775-4275-b6b0-1415834aae67 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:53.753958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c2b39396-b775-4275-b6b0-1415834aae67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:53.782427 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:53.826276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 508/1003] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:53.850314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25266e63-2fa7-4f9e-8f4c-c2f595a5883f None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:53.958865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25266e63-2fa7-4f9e-8f4c-c2f595a5883f None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:53.993922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fda330-3f40-4d9b-9e70-134a0e5d0206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:53.994483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fda330-3f40-4d9b-9e70-134a0e5d0206 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:53.994756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fda330-3f40-4d9b-9e70-134a0e5d0206 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:53.995019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fda330-3f40-4d9b-9e70-134a0e5d0206 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:53.995644 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fda330-3f40-4d9b-9e70-134a0e5d0206 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:53.997040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fda330-3f40-4d9b-9e70-134a0e5d0206 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-20fda330-3f40-4d9b-9e70-134a0e5d0206 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:54.006394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:54.007398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:54.008224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 496/1004] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Aug 1 11:51:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:54.016328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b39396-b775-4275-b6b0-1415834aae67 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:54.125497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b39396-b775-4275-b6b0-1415834aae67 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:54.159120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf6987-c66f-4c68-b511-88ec48509dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:54.159745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf6987-c66f-4c68-b511-88ec48509dfd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:54.160024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf6987-c66f-4c68-b511-88ec48509dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:54.160303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf6987-c66f-4c68-b511-88ec48509dfd None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:54.161007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf6987-c66f-4c68-b511-88ec48509dfd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:54.161307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf6987-c66f-4c68-b511-88ec48509dfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:54.162686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf6987-c66f-4c68-b511-88ec48509dfd None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:54.167740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 509/1005] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Aug 1 11:51:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:54.175319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fda330-3f40-4d9b-9e70-134a0e5d0206 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:54.237223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:54.242584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6678b-a320-4c79-9184-5f9258aaa058 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:54.242962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6678b-a320-4c79-9184-5f9258aaa058 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:54.243305 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6678b-a320-4c79-9184-5f9258aaa058 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:54.248405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 510/1006] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:54.250194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45407d-2242-4d40-a1b4-828832655c6d None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:54.250642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45407d-2242-4d40-a1b4-828832655c6d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:54.250917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45407d-2242-4d40-a1b4-828832655c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:54.269510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8e45407d-2242-4d40-a1b4-828832655c6d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:54.270702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8e45407d-2242-4d40-a1b4-828832655c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:54.284577 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fda330-3f40-4d9b-9e70-134a0e5d0206 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:54.296276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:54.320904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b805a-5f6a-417f-aa1e-ea039770661e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:54.321532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b805a-5f6a-417f-aa1e-ea039770661e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:54.321817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b805a-5f6a-417f-aa1e-ea039770661e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:54.322087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b805a-5f6a-417f-aa1e-ea039770661e None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:54.322769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b805a-5f6a-417f-aa1e-ea039770661e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:54.323073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b805a-5f6a-417f-aa1e-ea039770661e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:54.324685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b805a-5f6a-417f-aa1e-ea039770661e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:54.335006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 497/1007] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Aug 1 11:51:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:54.335739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b805a-5f6a-417f-aa1e-ea039770661e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:54.347320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 511/1008] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:54.369569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45407d-2242-4d40-a1b4-828832655c6d None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:54.445954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b805a-5f6a-417f-aa1e-ea039770661e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:54.470565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:54.480715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45407d-2242-4d40-a1b4-828832655c6d None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:54.483970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd9627-a091-4f34-967a-ea8ea52d7aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:54.484635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd9627-a091-4f34-967a-ea8ea52d7aac None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:54.484917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd9627-a091-4f34-967a-ea8ea52d7aac None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:54.485203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd9627-a091-4f34-967a-ea8ea52d7aac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:54.486022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd9627-a091-4f34-967a-ea8ea52d7aac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:54.487461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd9627-a091-4f34-967a-ea8ea52d7aac None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:54.516593 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 498/1009] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 11:51:54 2022] POST /identity/v3/endpoints => generated 325 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 11:51:54.524709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ec9a9-fe0d-410b-957b-8d94e18ebfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:54.525372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ec9a9-fe0d-410b-957b-8d94e18ebfd5 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:54.525648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ec9a9-fe0d-410b-957b-8d94e18ebfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:54.525909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ec9a9-fe0d-410b-957b-8d94e18ebfd5 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:54.526632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ec9a9-fe0d-410b-957b-8d94e18ebfd5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:54.528058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ec9a9-fe0d-410b-957b-8d94e18ebfd5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:54.547058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 512/1010] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:51:54 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 11:51:54.968511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a94e2-1f52-4c5d-80f4-5766fe2dec94 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:54.969056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a94e2-1f52-4c5d-80f4-5766fe2dec94 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:54.969056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a94e2-1f52-4c5d-80f4-5766fe2dec94 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:54.970664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 499/1011] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:54.975864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e6ab8-8b32-4d6b-87c4-06e1c5dff44a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:54.976340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e6ab8-8b32-4d6b-87c4-06e1c5dff44a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:54.976420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e6ab8-8b32-4d6b-87c4-06e1c5dff44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:54.998254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-696e6ab8-8b32-4d6b-87c4-06e1c5dff44a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:54.999461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-696e6ab8-8b32-4d6b-87c4-06e1c5dff44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:55.010216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:55.078631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 513/1012] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:54 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:55.100908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a94e2-1f52-4c5d-80f4-5766fe2dec94 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:55.211026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a94e2-1f52-4c5d-80f4-5766fe2dec94 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:55.246735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba87e3-955f-4236-8c80-8368e4116949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:55.247390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba87e3-955f-4236-8c80-8368e4116949 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:55.247670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba87e3-955f-4236-8c80-8368e4116949 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:55.247945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abba87e3-955f-4236-8c80-8368e4116949 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:55.248603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba87e3-955f-4236-8c80-8368e4116949 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:55.249984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abba87e3-955f-4236-8c80-8368e4116949 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-abba87e3-955f-4236-8c80-8368e4116949 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:55.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:55.260267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:55.266474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 500/1013] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:55.268863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e6ab8-8b32-4d6b-87c4-06e1c5dff44a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:55.371843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e6ab8-8b32-4d6b-87c4-06e1c5dff44a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:55.406391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668feeb-bdcc-47fb-a951-590664cf6aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:55.407024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668feeb-bdcc-47fb-a951-590664cf6aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:55.407304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668feeb-bdcc-47fb-a951-590664cf6aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:55.407637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668feeb-bdcc-47fb-a951-590664cf6aa4 None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:55.408346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668feeb-bdcc-47fb-a951-590664cf6aa4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:55.408654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668feeb-bdcc-47fb-a951-590664cf6aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:55.410045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668feeb-bdcc-47fb-a951-590664cf6aa4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:55.419820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 514/1014] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:55.421573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba87e3-955f-4236-8c80-8368e4116949 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:55.473576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:55.526795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abba87e3-955f-4236-8c80-8368e4116949 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:55.561729 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd4e75-8438-4c4e-a11d-5163f2cf5c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:55.562355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd4e75-8438-4c4e-a11d-5163f2cf5c3a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:55.562631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd4e75-8438-4c4e-a11d-5163f2cf5c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:55.562902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd4e75-8438-4c4e-a11d-5163f2cf5c3a None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:55.563587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd4e75-8438-4c4e-a11d-5163f2cf5c3a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:55.563926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd4e75-8438-4c4e-a11d-5163f2cf5c3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:55.565345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dd4e75-8438-4c4e-a11d-5163f2cf5c3a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:55.570549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 501/1015] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Aug 1 11:51:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:55.576142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668feeb-bdcc-47fb-a951-590664cf6aa4 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:55.580190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:55.689194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668feeb-bdcc-47fb-a951-590664cf6aa4 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:55.726013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb3ff3-b965-47c6-bd49-756753313f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:55.726639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb3ff3-b965-47c6-bd49-756753313f50 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:55.726917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb3ff3-b965-47c6-bd49-756753313f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:55.727184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb3ff3-b965-47c6-bd49-756753313f50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:55.727894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb3ff3-b965-47c6-bd49-756753313f50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:55.729313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb3ff3-b965-47c6-bd49-756753313f50 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:55.752173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 515/1016] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 11:51:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:56.542525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560c890-72e9-4484-a82a-6569da6cb730 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:56.543080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560c890-72e9-4484-a82a-6569da6cb730 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:56.543080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560c890-72e9-4484-a82a-6569da6cb730 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:56.544673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 502/1017] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:56.550174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16adb8d-58a9-4629-9159-19ddfc59d77f None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:56.550630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16adb8d-58a9-4629-9159-19ddfc59d77f None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:56.550711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16adb8d-58a9-4629-9159-19ddfc59d77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:56.561263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664522-1df5-48dd-873a-dfda903e7657 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:56.561563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664522-1df5-48dd-873a-dfda903e7657 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:56.561825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664522-1df5-48dd-873a-dfda903e7657 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:56.563335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 503/1018] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:51:56.568279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be838e5-6e76-4458-a543-2b94a9e248ac None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:56.568594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be838e5-6e76-4458-a543-2b94a9e248ac None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:56.568872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be838e5-6e76-4458-a543-2b94a9e248ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:56.570422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c16adb8d-58a9-4629-9159-19ddfc59d77f None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:56.571647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c16adb8d-58a9-4629-9159-19ddfc59d77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:56.587208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-8be838e5-6e76-4458-a543-2b94a9e248ac None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:56.588456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8be838e5-6e76-4458-a543-2b94a9e248ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:56.636319 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:56.656338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 516/1019] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:56.662680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 504/1020] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:56.678940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16adb8d-58a9-4629-9159-19ddfc59d77f None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:56.684618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be838e5-6e76-4458-a543-2b94a9e248ac None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:56.730592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:56.786622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16adb8d-58a9-4629-9159-19ddfc59d77f None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:56.792442 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be838e5-6e76-4458-a543-2b94a9e248ac None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:56.823906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784a5a1-c3f1-4b71-84ba-3557f864112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:56.824809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784a5a1-c3f1-4b71-84ba-3557f864112c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:56.825226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784a5a1-c3f1-4b71-84ba-3557f864112c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:56.825504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784a5a1-c3f1-4b71-84ba-3557f864112c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:56.826676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784a5a1-c3f1-4b71-84ba-3557f864112c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:56.829134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32b7e3-384e-40ca-add1-840a9c05c9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:56.829845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32b7e3-384e-40ca-add1-840a9c05c9b9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:56.830128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32b7e3-384e-40ca-add1-840a9c05c9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:56.830404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32b7e3-384e-40ca-add1-840a9c05c9b9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:56.831028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32b7e3-384e-40ca-add1-840a9c05c9b9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:56.831667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784a5a1-c3f1-4b71-84ba-3557f864112c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:56.832433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32b7e3-384e-40ca-add1-840a9c05c9b9 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:56.838277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 517/1021] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:56.839090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784a5a1-c3f1-4b71-84ba-3557f864112c None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-de32b7e3-384e-40ca-add1-840a9c05c9b9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:56.842249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:56.843216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:56.845319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 505/1022] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Aug 1 11:51:56 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:56.849812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32b7e3-384e-40ca-add1-840a9c05c9b9 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:56.906459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:56.957859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784a5a1-c3f1-4b71-84ba-3557f864112c None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:56.961951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32b7e3-384e-40ca-add1-840a9c05c9b9 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:56.969461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:56.996148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b950221-908d-4ba5-964a-86242e336bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:56.996813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b950221-908d-4ba5-964a-86242e336bec None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:56.997133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b950221-908d-4ba5-964a-86242e336bec None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:56.997364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b950221-908d-4ba5-964a-86242e336bec None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:56.998066 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b950221-908d-4ba5-964a-86242e336bec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:56.999513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b950221-908d-4ba5-964a-86242e336bec None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:56.999626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca13ab-c837-412b-8177-22865c4a6f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:57.000231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca13ab-c837-412b-8177-22865c4a6f6f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:57.000541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca13ab-c837-412b-8177-22865c4a6f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:57.000814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca13ab-c837-412b-8177-22865c4a6f6f None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:57.001532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca13ab-c837-412b-8177-22865c4a6f6f None admin] RBAC: Adding query filter params (name=image) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:57.001836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca13ab-c837-412b-8177-22865c4a6f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:57.003195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca13ab-c837-412b-8177-22865c4a6f6f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:57.013851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 506/1023] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:57.014575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca13ab-c837-412b-8177-22865c4a6f6f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:57.044313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 518/1024] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:51:56 2022] POST /identity/v3/projects => generated 311 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:51:57.138646 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca13ab-c837-412b-8177-22865c4a6f6f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:57.175506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f0baa-828a-44de-8310-5a8e22bf3be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:57.176146 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f0baa-828a-44de-8310-5a8e22bf3be6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:57.176456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f0baa-828a-44de-8310-5a8e22bf3be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:57.176729 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f0baa-828a-44de-8310-5a8e22bf3be6 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:57.177423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f0baa-828a-44de-8310-5a8e22bf3be6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:57.177728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f0baa-828a-44de-8310-5a8e22bf3be6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:57.179128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f0baa-828a-44de-8310-5a8e22bf3be6 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:57.184705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 507/1025] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:57.191605 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b950221-908d-4ba5-964a-86242e336bec None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:57.298542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b950221-908d-4ba5-964a-86242e336bec None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:57.306005 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:57.335807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a3b70-ae12-4dd3-a070-d29a7c84d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:57.336465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a3b70-ae12-4dd3-a070-d29a7c84d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:57.336751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a3b70-ae12-4dd3-a070-d29a7c84d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:57.337022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a3b70-ae12-4dd3-a070-d29a7c84d5c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:57.337788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a3b70-ae12-4dd3-a070-d29a7c84d5c5 None admin] RBAC: Adding query filter params (interface=public, service_id=09741cd14ebd439fbbe3441e8e316716, region_id=RegionOne) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:57.338090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a3b70-ae12-4dd3-a070-d29a7c84d5c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:57.339475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a3b70-ae12-4dd3-a070-d29a7c84d5c5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:57.345269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 519/1026] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Aug 1 11:51:57 2022] GET /identity/v3/endpoints?service_id=09741cd14ebd439fbbe3441e8e316716&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:57.350799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f0baa-828a-44de-8310-5a8e22bf3be6 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:57.387901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:57.460169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f0baa-828a-44de-8310-5a8e22bf3be6 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:57.498109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727f616-c881-4443-a92f-fd896c77edca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:57.498753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727f616-c881-4443-a92f-fd896c77edca None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:57.499034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727f616-c881-4443-a92f-fd896c77edca None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:57.499297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727f616-c881-4443-a92f-fd896c77edca None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:57.499969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727f616-c881-4443-a92f-fd896c77edca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:57.501515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727f616-c881-4443-a92f-fd896c77edca None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:57.516331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 508/1027] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Aug 1 11:51:57 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 11:51:57.844672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10117-6167-41c4-8d6d-8af8aa57a677 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:57.845895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10117-6167-41c4-8d6d-8af8aa57a677 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:57.846395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b10117-6167-41c4-8d6d-8af8aa57a677 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:57.847987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 520/1028] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:57.853886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e95596-7f0f-4b7b-a7f6-a0926dae29b2 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:57.854297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e95596-7f0f-4b7b-a7f6-a0926dae29b2 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:57.854476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e95596-7f0f-4b7b-a7f6-a0926dae29b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:57.873571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-14e95596-7f0f-4b7b-a7f6-a0926dae29b2 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:57.874761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-14e95596-7f0f-4b7b-a7f6-a0926dae29b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:57.943967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 509/1029] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:51:57.966551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10117-6167-41c4-8d6d-8af8aa57a677 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:58.083343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b10117-6167-41c4-8d6d-8af8aa57a677 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:58.121508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f76cc8-9f23-4794-b805-c9702d72f633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:58.122509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f76cc8-9f23-4794-b805-c9702d72f633 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:58.123007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f76cc8-9f23-4794-b805-c9702d72f633 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:58.123573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f76cc8-9f23-4794-b805-c9702d72f633 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:58.134619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f76cc8-9f23-4794-b805-c9702d72f633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-90f76cc8-9f23-4794-b805-c9702d72f633 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:51:58.139857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:58.141004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:51:58.142076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:58.144459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 521/1030] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Aug 1 11:51:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:58.150393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e95596-7f0f-4b7b-a7f6-a0926dae29b2 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:58.180878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:58.258150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e95596-7f0f-4b7b-a7f6-a0926dae29b2 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:58.296554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892d9bf-3d42-41f1-93a2-680f89053aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:58.297192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892d9bf-3d42-41f1-93a2-680f89053aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:58.297462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892d9bf-3d42-41f1-93a2-680f89053aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:58.297796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892d9bf-3d42-41f1-93a2-680f89053aa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:58.298489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892d9bf-3d42-41f1-93a2-680f89053aa9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:58.298789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892d9bf-3d42-41f1-93a2-680f89053aa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:58.300167 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892d9bf-3d42-41f1-93a2-680f89053aa9 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:58.308793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 510/1031] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Aug 1 11:51:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:58.314508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f76cc8-9f23-4794-b805-c9702d72f633 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:58.345890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:58.417849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f76cc8-9f23-4794-b805-c9702d72f633 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:58.451997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db3a2b-500a-439e-b69c-6b50735fe644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:58.452676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db3a2b-500a-439e-b69c-6b50735fe644 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:58.452951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db3a2b-500a-439e-b69c-6b50735fe644 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:58.453226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db3a2b-500a-439e-b69c-6b50735fe644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:58.463572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db3a2b-500a-439e-b69c-6b50735fe644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:58.465054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db3a2b-500a-439e-b69c-6b50735fe644 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-13db3a2b-500a-439e-b69c-6b50735fe644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:58.479784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:58.480749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:58.481644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:58.481644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:58.481644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:58.481644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:58.482504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 522/1032] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:58.487203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892d9bf-3d42-41f1-93a2-680f89053aa9 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:58.585796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:58.598654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892d9bf-3d42-41f1-93a2-680f89053aa9 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:58.636130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782f831-9524-4293-9f4f-16f7a3d642b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:58.636808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782f831-9524-4293-9f4f-16f7a3d642b3 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:58.637089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782f831-9524-4293-9f4f-16f7a3d642b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:58.637358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d782f831-9524-4293-9f4f-16f7a3d642b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:58.638056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782f831-9524-4293-9f4f-16f7a3d642b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:58.638360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782f831-9524-4293-9f4f-16f7a3d642b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:58.639738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d782f831-9524-4293-9f4f-16f7a3d642b3 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:58.652544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 511/1033] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:51:58.658151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db3a2b-500a-439e-b69c-6b50735fe644 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:58.746251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:58.767454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db3a2b-500a-439e-b69c-6b50735fe644 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:58.805301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485c565-abfd-46e6-99ca-e108cc16929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:58.805942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485c565-abfd-46e6-99ca-e108cc16929a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:58.806218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485c565-abfd-46e6-99ca-e108cc16929a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:58.806491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485c565-abfd-46e6-99ca-e108cc16929a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:58.814547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485c565-abfd-46e6-99ca-e108cc16929a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:58.816013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485c565-abfd-46e6-99ca-e108cc16929a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-a485c565-abfd-46e6-99ca-e108cc16929a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:51:58.827729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:58.828969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:58.835023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 523/1034] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Aug 1 11:51:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:58.837219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782f831-9524-4293-9f4f-16f7a3d642b3 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:58.949877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d782f831-9524-4293-9f4f-16f7a3d642b3 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:58.986494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b11ad-f99e-45b4-9a1d-0c35f6324b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:58.987124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b11ad-f99e-45b4-9a1d-0c35f6324b44 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:58.987404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b11ad-f99e-45b4-9a1d-0c35f6324b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:58.987679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b11ad-f99e-45b4-9a1d-0c35f6324b44 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:58.988540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b11ad-f99e-45b4-9a1d-0c35f6324b44 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:58.988849 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b11ad-f99e-45b4-9a1d-0c35f6324b44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:58.990307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b11ad-f99e-45b4-9a1d-0c35f6324b44 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:59.009658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 512/1035] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Aug 1 11:51:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:59.015228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485c565-abfd-46e6-99ca-e108cc16929a None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:59.082228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe2a63-cec6-423c-bbf1-08d6d66c3c80 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.082575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe2a63-cec6-423c-bbf1-08d6d66c3c80 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.082766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe2a63-cec6-423c-bbf1-08d6d66c3c80 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.084176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 513/1036] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:59.089576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7cc0a-9550-4eb5-9ff5-e2ce28a65e8b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.089852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7cc0a-9550-4eb5-9ff5-e2ce28a65e8b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.090118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7cc0a-9550-4eb5-9ff5-e2ce28a65e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.108695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-fdf7cc0a-9550-4eb5-9ff5-e2ce28a65e8b None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:59.109906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fdf7cc0a-9550-4eb5-9ff5-e2ce28a65e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:59.112030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:59.123819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485c565-abfd-46e6-99ca-e108cc16929a None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:59.164936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03107c29-ca23-4aa1-bfc5-76f79305f678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:59.165908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03107c29-ca23-4aa1-bfc5-76f79305f678 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.166404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03107c29-ca23-4aa1-bfc5-76f79305f678 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.166873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03107c29-ca23-4aa1-bfc5-76f79305f678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.179920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:59.182512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 514/1037] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:59.206239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7cc0a-9550-4eb5-9ff5-e2ce28a65e8b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:59.241291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03107c29-ca23-4aa1-bfc5-76f79305f678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:59.241766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03107c29-ca23-4aa1-bfc5-76f79305f678 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:59.311538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7cc0a-9550-4eb5-9ff5-e2ce28a65e8b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:59.348134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f167b2-edca-465d-8aa5-3487782ad91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:59.348861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f167b2-edca-465d-8aa5-3487782ad91d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.349211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f167b2-edca-465d-8aa5-3487782ad91d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.349504 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f167b2-edca-465d-8aa5-3487782ad91d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.350800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f167b2-edca-465d-8aa5-3487782ad91d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:59.352481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f167b2-edca-465d-8aa5-3487782ad91d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:59.354118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 515/1038] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:59.360185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f167b2-edca-465d-8aa5-3487782ad91d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:59.363048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 524/1039] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:51:59 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:51:59.368645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03107c29-ca23-4aa1-bfc5-76f79305f678 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:59.369237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:59.422844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:59.466065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f167b2-edca-465d-8aa5-3487782ad91d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:59.489920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03107c29-ca23-4aa1-bfc5-76f79305f678 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:59.503288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbb549-c93a-4a4a-93e8-a1e8f79390f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:59.504046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbb549-c93a-4a4a-93e8-a1e8f79390f0 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.504351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbb549-c93a-4a4a-93e8-a1e8f79390f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.504659 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbb549-c93a-4a4a-93e8-a1e8f79390f0 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.505335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbb549-c93a-4a4a-93e8-a1e8f79390f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:59.506803 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbb549-c93a-4a4a-93e8-a1e8f79390f0 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:59.529340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9d0ab-0822-4e4e-9859-ab40b74a9122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:59.530007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9d0ab-0822-4e4e-9859-ab40b74a9122 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.530286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9d0ab-0822-4e4e-9859-ab40b74a9122 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.530503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 516/1040] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:51:59 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 11:51:59.530631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9d0ab-0822-4e4e-9859-ab40b74a9122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.531806 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d5e86-e258-4f66-bc2f-c17ba11e9e1b None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.532075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d5e86-e258-4f66-bc2f-c17ba11e9e1b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.532697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d5e86-e258-4f66-bc2f-c17ba11e9e1b None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.534692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 517/1041] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:51:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:51:59.540162 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc7508-aab4-4e01-976e-25da2f5315b5 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.540548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc7508-aab4-4e01-976e-25da2f5315b5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.540760 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc7508-aab4-4e01-976e-25da2f5315b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.545153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9d0ab-0822-4e4e-9859-ab40b74a9122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:59.546836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9d0ab-0822-4e4e-9859-ab40b74a9122 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:59.560519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-41dc7508-aab4-4e01-976e-25da2f5315b5 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:51:59.561827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-41dc7508-aab4-4e01-976e-25da2f5315b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-33f9d0ab-0822-4e4e-9859-ab40b74a9122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:51:59.564543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:59.565454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:59.566366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:51:59.566366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:51:59.566366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:51:59.566366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:59.566366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 525/1042] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:59.571809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9d0ab-0822-4e4e-9859-ab40b74a9122 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:59.633841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:59.649445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 518/1043] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:51:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:51:59.672277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc7508-aab4-4e01-976e-25da2f5315b5 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:59.681525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:59.691538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9d0ab-0822-4e4e-9859-ab40b74a9122 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:59.731542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c2c1c-892b-44db-aaf8-1db36ec8cee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:59.733225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c2c1c-892b-44db-aaf8-1db36ec8cee6 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.733225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c2c1c-892b-44db-aaf8-1db36ec8cee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.733225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c2c1c-892b-44db-aaf8-1db36ec8cee6 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.734185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c2c1c-892b-44db-aaf8-1db36ec8cee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:51:59.734575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c2c1c-892b-44db-aaf8-1db36ec8cee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:59.736082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c2c1c-892b-44db-aaf8-1db36ec8cee6 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:59.749340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 526/1044] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:51:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:51:59.758444 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c2c1c-892b-44db-aaf8-1db36ec8cee6 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:59.785049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc7508-aab4-4e01-976e-25da2f5315b5 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:59.822746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e94a3-02c9-486f-95b5-ee90c047e4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:59.823357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e94a3-02c9-486f-95b5-ee90c047e4dc None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.823604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e94a3-02c9-486f-95b5-ee90c047e4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.823878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e94a3-02c9-486f-95b5-ee90c047e4dc None admin] PATH_INFO: `/v3/services/image` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.824572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e94a3-02c9-486f-95b5-ee90c047e4dc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:59.825954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e94a3-02c9-486f-95b5-ee90c047e4dc None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-0d0e94a3-02c9-486f-95b5-ee90c047e4dc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:59.837642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:59.838825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:51:59.839829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 519/1045] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Aug 1 11:51:59 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:51:59.844907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e94a3-02c9-486f-95b5-ee90c047e4dc None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:59.873772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c2c1c-892b-44db-aaf8-1db36ec8cee6 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:51:59.909442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc1876-63c2-460c-b032-a026c09e0084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:51:59.910246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc1876-63c2-460c-b032-a026c09e0084 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:51:59.910620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc1876-63c2-460c-b032-a026c09e0084 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:51:59.910990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc1876-63c2-460c-b032-a026c09e0084 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:51:59.913394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:59.918668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc1876-63c2-460c-b032-a026c09e0084 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:51:59.920133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc1876-63c2-460c-b032-a026c09e0084 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-d5dc1876-63c2-460c-b032-a026c09e0084 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:51:59.931821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:51:59.932998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:51:59.933899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:51:59.933899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:51:59.933899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:51:59.933899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:51:59.933899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:51:59.933899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:51:59.933899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:51:59.933899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:51:59.937948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 527/1046] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:51:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:51:59.938756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc1876-63c2-460c-b032-a026c09e0084 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:51:59.956811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:51:59.963616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e94a3-02c9-486f-95b5-ee90c047e4dc None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:00.001279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553de6b-d7bb-4bfd-b4a5-aa586dd684b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:00.001937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553de6b-d7bb-4bfd-b4a5-aa586dd684b9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:00.002222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553de6b-d7bb-4bfd-b4a5-aa586dd684b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:00.002501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553de6b-d7bb-4bfd-b4a5-aa586dd684b9 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:00.003201 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553de6b-d7bb-4bfd-b4a5-aa586dd684b9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:00.003638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553de6b-d7bb-4bfd-b4a5-aa586dd684b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:00.004969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553de6b-d7bb-4bfd-b4a5-aa586dd684b9 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:00.015445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 520/1047] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:51:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:00.016831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553de6b-d7bb-4bfd-b4a5-aa586dd684b9 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:00.060068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc1876-63c2-460c-b032-a026c09e0084 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:00.098400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fdca1-11fa-46f5-884d-0642b9f77d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:00.099033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fdca1-11fa-46f5-884d-0642b9f77d5b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:00.099309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fdca1-11fa-46f5-884d-0642b9f77d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:00.099582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fdca1-11fa-46f5-884d-0642b9f77d5b None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:00.100803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fdca1-11fa-46f5-884d-0642b9f77d5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:00.101231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fdca1-11fa-46f5-884d-0642b9f77d5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:00.103039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fdca1-11fa-46f5-884d-0642b9f77d5b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:00.115951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 528/1048] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:51:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:00.121778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fdca1-11fa-46f5-884d-0642b9f77d5b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:00.127091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553de6b-d7bb-4bfd-b4a5-aa586dd684b9 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:00.150109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:00.165521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928dac2-03da-4499-aa71-b14eefca2896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:00.166156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928dac2-03da-4499-aa71-b14eefca2896 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:00.166440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928dac2-03da-4499-aa71-b14eefca2896 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:00.166717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928dac2-03da-4499-aa71-b14eefca2896 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:00.167405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928dac2-03da-4499-aa71-b14eefca2896 None admin] RBAC: Adding query filter params (type=image) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:00.167712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928dac2-03da-4499-aa71-b14eefca2896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:00.169132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928dac2-03da-4499-aa71-b14eefca2896 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:00.174399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 521/1049] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:52:00 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:00.180065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928dac2-03da-4499-aa71-b14eefca2896 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:00.204589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:00.232243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fdca1-11fa-46f5-884d-0642b9f77d5b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:00.271693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51776694-9948-49c8-9180-ef86858c290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:00.272674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51776694-9948-49c8-9180-ef86858c290b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:00.273033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51776694-9948-49c8-9180-ef86858c290b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:00.273310 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51776694-9948-49c8-9180-ef86858c290b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:00.274171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51776694-9948-49c8-9180-ef86858c290b None admin] RBAC: Adding query filter params (role.id=9a4d5a7314884a599e1b22288a1add9d, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=db3ff1b21c784e8cb5bffcb79f230d08) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:00.274484 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51776694-9948-49c8-9180-ef86858c290b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:00.275968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51776694-9948-49c8-9180-ef86858c290b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:00.282033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 529/1050] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:52:00 2022] GET /identity/v3/role_assignments?user.id=db3ff1b21c784e8cb5bffcb79f230d08&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=9a4d5a7314884a599e1b22288a1add9d => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:00.283431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928dac2-03da-4499-aa71-b14eefca2896 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:00.322205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e4780-c804-43e9-b7c0-8bd8aefdf203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:00.323247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e4780-c804-43e9-b7c0-8bd8aefdf203 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:00.323773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e4780-c804-43e9-b7c0-8bd8aefdf203 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:00.324239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e4780-c804-43e9-b7c0-8bd8aefdf203 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:00.325265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e4780-c804-43e9-b7c0-8bd8aefdf203 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:00.326908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e4780-c804-43e9-b7c0-8bd8aefdf203 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:00.353499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 522/1051] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 11:52:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:52:01.613381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995f024-d4e4-4311-850d-fa96a4fd3840 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:01.613988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995f024-d4e4-4311-850d-fa96a4fd3840 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:01.613988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995f024-d4e4-4311-850d-fa96a4fd3840 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:01.615457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 530/1052] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:01.620762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07364f1f-484e-42cb-8c57-3223951f33e0 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:01.621176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07364f1f-484e-42cb-8c57-3223951f33e0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:01.621344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07364f1f-484e-42cb-8c57-3223951f33e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:01.639805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-07364f1f-484e-42cb-8c57-3223951f33e0 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:01.641035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-07364f1f-484e-42cb-8c57-3223951f33e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:01.676385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:01.717151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 523/1053] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:01.741241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995f024-d4e4-4311-850d-fa96a4fd3840 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:01.869778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995f024-d4e4-4311-850d-fa96a4fd3840 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:01.906579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a980ade-b7fd-4125-8223-af682351b372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:01.907263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a980ade-b7fd-4125-8223-af682351b372 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:01.907519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a980ade-b7fd-4125-8223-af682351b372 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:01.907794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a980ade-b7fd-4125-8223-af682351b372 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:01.915009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:01.919397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a980ade-b7fd-4125-8223-af682351b372 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-2a980ade-b7fd-4125-8223-af682351b372 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:01.925236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:01.926168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:01.927014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:52:01.931470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 531/1054] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:52:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:01.932785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07364f1f-484e-42cb-8c57-3223951f33e0 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:02.037742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07364f1f-484e-42cb-8c57-3223951f33e0 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:02.073733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914dd06-f6ad-47d5-b1e0-51f5f4fd76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:02.074366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914dd06-f6ad-47d5-b1e0-51f5f4fd76c9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.074642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914dd06-f6ad-47d5-b1e0-51f5f4fd76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.074909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914dd06-f6ad-47d5-b1e0-51f5f4fd76c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.075625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914dd06-f6ad-47d5-b1e0-51f5f4fd76c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:02.075922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914dd06-f6ad-47d5-b1e0-51f5f4fd76c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:02.077326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914dd06-f6ad-47d5-b1e0-51f5f4fd76c9 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:02.086599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 524/1055] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:52:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:02.092472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a980ade-b7fd-4125-8223-af682351b372 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:02.199054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a980ade-b7fd-4125-8223-af682351b372 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:02.235243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af3db4-9ddc-4a68-810a-6a17b0aa7e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:02.235901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af3db4-9ddc-4a68-810a-6a17b0aa7e0f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.236183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af3db4-9ddc-4a68-810a-6a17b0aa7e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.236483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af3db4-9ddc-4a68-810a-6a17b0aa7e0f None admin] PATH_INFO: `/v3/users/dd3d609a83044b17ad19c08a0eb93045` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.251877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af3db4-9ddc-4a68-810a-6a17b0aa7e0f None admin] RBAC: Authorizing `identity:get_user(user_id=dd3d609a83044b17ad19c08a0eb93045)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:02.253413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af3db4-9ddc-4a68-810a-6a17b0aa7e0f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:02.259242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 532/1056] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:02 2022] GET /identity/v3/users/dd3d609a83044b17ad19c08a0eb93045 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:02.259392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581e8a3-b1cd-42b4-87c2-0272481b4fe3 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.259392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581e8a3-b1cd-42b4-87c2-0272481b4fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.259392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581e8a3-b1cd-42b4-87c2-0272481b4fe3 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.259392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 525/1057] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:52:02.261801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af3db4-9ddc-4a68-810a-6a17b0aa7e0f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:02.263921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c5ffd-7c59-439c-b9aa-3d69855a7401 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.264206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c5ffd-7c59-439c-b9aa-3d69855a7401 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.264516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c5ffd-7c59-439c-b9aa-3d69855a7401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.282646 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ed9c5ffd-7c59-439c-b9aa-3d69855a7401 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:02.283855 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ed9c5ffd-7c59-439c-b9aa-3d69855a7401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:02.309476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:02.326701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:02.361067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 526/1058] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:52:02.374290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196907bd-c8ba-49a0-bcf1-ad6eec1535db None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.374570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196907bd-c8ba-49a0-bcf1-ad6eec1535db None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.374840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196907bd-c8ba-49a0-bcf1-ad6eec1535db None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.376073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af3db4-9ddc-4a68-810a-6a17b0aa7e0f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:02.376516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 527/1059] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:52:02.382344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48425bcd-576f-4260-9c48-811295fdc0e8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.382615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48425bcd-576f-4260-9c48-811295fdc0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.382879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48425bcd-576f-4260-9c48-811295fdc0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.401506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-48425bcd-576f-4260-9c48-811295fdc0e8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:02.403183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-48425bcd-576f-4260-9c48-811295fdc0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:02.416674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592b3c3-b314-4e69-aa17-f7279031dcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:02.417647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592b3c3-b314-4e69-aa17-f7279031dcbe None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.417972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592b3c3-b314-4e69-aa17-f7279031dcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.418376 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592b3c3-b314-4e69-aa17-f7279031dcbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.475479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 528/1060] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:52:02.476247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48425bcd-576f-4260-9c48-811295fdc0e8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:02.489948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592b3c3-b314-4e69-aa17-f7279031dcbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:02.491517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592b3c3-b314-4e69-aa17-f7279031dcbe None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:02.550447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:02.565069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:02.577261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48425bcd-576f-4260-9c48-811295fdc0e8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:02.598262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 533/1061] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:02.599510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592b3c3-b314-4e69-aa17-f7279031dcbe None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:02.614377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb1785-b782-4ce4-80f2-e32700efcf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:02.615038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb1785-b782-4ce4-80f2-e32700efcf31 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.615316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb1785-b782-4ce4-80f2-e32700efcf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.615595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb1785-b782-4ce4-80f2-e32700efcf31 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.626515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb1785-b782-4ce4-80f2-e32700efcf31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-24cb1785-b782-4ce4-80f2-e32700efcf31 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:02.631536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:02.632559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:02.633385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 529/1062] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Aug 1 11:52:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:02.634064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb1785-b782-4ce4-80f2-e32700efcf31 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:02.702722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592b3c3-b314-4e69-aa17-f7279031dcbe None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:02.736793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb1785-b782-4ce4-80f2-e32700efcf31 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:02.738798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da5987-128c-4d78-b204-f00c08c17db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:02.739439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da5987-128c-4d78-b204-f00c08c17db7 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.739718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da5987-128c-4d78-b204-f00c08c17db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.739988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da5987-128c-4d78-b204-f00c08c17db7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.770498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:02.776275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc6edd-8102-4e51-9ac2-fa7458280df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:02.776952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc6edd-8102-4e51-9ac2-fa7458280df4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.777237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc6edd-8102-4e51-9ac2-fa7458280df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.777528 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc6edd-8102-4e51-9ac2-fa7458280df4 None admin] PATH_INFO: `/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.778871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc6edd-8102-4e51-9ac2-fa7458280df4 None admin] RBAC: Authorizing `identity:get_project(project_id=1c61c0f1dd8b47969fad7c1f653bf27c)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:02.780323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc6edd-8102-4e51-9ac2-fa7458280df4 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:02.782072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 530/1063] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:02 2022] GET /identity/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:02.782930 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc6edd-8102-4e51-9ac2-fa7458280df4 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:02.805021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:02.818776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da5987-128c-4d78-b204-f00c08c17db7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:02.820281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da5987-128c-4d78-b204-f00c08c17db7 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:02.900514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc6edd-8102-4e51-9ac2-fa7458280df4 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:02.927021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 534/1064] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:02.927873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da5987-128c-4d78-b204-f00c08c17db7 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:02.936846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757972-37d7-4872-b50b-44757f0dc680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:02.937480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757972-37d7-4872-b50b-44757f0dc680 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:02.937760 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757972-37d7-4872-b50b-44757f0dc680 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:02.938037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757972-37d7-4872-b50b-44757f0dc680 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:02.938826 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd757972-37d7-4872-b50b-44757f0dc680 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:02.939136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd757972-37d7-4872-b50b-44757f0dc680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:02.940543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd757972-37d7-4872-b50b-44757f0dc680 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:02.948821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 531/1065] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Aug 1 11:52:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:02.949645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757972-37d7-4872-b50b-44757f0dc680 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:03.001524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:03.030736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:03.046857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da5987-128c-4d78-b204-f00c08c17db7 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:03.053859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757972-37d7-4872-b50b-44757f0dc680 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:03.087946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6df933-31f3-41c5-abf8-bfb346a5ebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:03.089114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6df933-31f3-41c5-abf8-bfb346a5ebd2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:03.089631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6df933-31f3-41c5-abf8-bfb346a5ebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:03.090103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6df933-31f3-41c5-abf8-bfb346a5ebd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:03.091468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6df933-31f3-41c5-abf8-bfb346a5ebd2 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=1c61c0f1dd8b47969fad7c1f653bf27c, user.id=dd3d609a83044b17ad19c08a0eb93045) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:03.091780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6df933-31f3-41c5-abf8-bfb346a5ebd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:03.093280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6df933-31f3-41c5-abf8-bfb346a5ebd2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:03.095413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f3492-5d4b-4244-a1ff-4b5900e73d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:03.096082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f3492-5d4b-4244-a1ff-4b5900e73d9e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:03.096756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f3492-5d4b-4244-a1ff-4b5900e73d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:03.096756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f3492-5d4b-4244-a1ff-4b5900e73d9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:03.099377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 535/1066] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:52:02 2022] GET /identity/v3/role_assignments?user.id=dd3d609a83044b17ad19c08a0eb93045&scope.project.id=1c61c0f1dd8b47969fad7c1f653bf27c&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:03.100626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6df933-31f3-41c5-abf8-bfb346a5ebd2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:03.116384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f3492-5d4b-4244-a1ff-4b5900e73d9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:03.117435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f3492-5d4b-4244-a1ff-4b5900e73d9e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-3e3f3492-5d4b-4244-a1ff-4b5900e73d9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:52:03.134195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:03.135407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:03.136336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:52:03.136336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:52:03.136336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:52:03.136336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:03.136336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 532/1067] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:03.142009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f3492-5d4b-4244-a1ff-4b5900e73d9e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:03.234304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6df933-31f3-41c5-abf8-bfb346a5ebd2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:03.258531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:03.275724 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5363cc-f02e-41ca-bd94-c56b16015a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:03.276420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5363cc-f02e-41ca-bd94-c56b16015a3e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:03.276701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5363cc-f02e-41ca-bd94-c56b16015a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:03.276970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5363cc-f02e-41ca-bd94-c56b16015a3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:03.287968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5363cc-f02e-41ca-bd94-c56b16015a3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:03.289469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5363cc-f02e-41ca-bd94-c56b16015a3e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-cb5363cc-f02e-41ca-bd94-c56b16015a3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:52:03.305211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:03.306155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:03.307038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:52:03.307038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:52:03.307038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:52:03.307038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:52:03.307038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 536/1068] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:03.313030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5363cc-f02e-41ca-bd94-c56b16015a3e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:03.318638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f3492-5d4b-4244-a1ff-4b5900e73d9e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:03.350257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:03.356848 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d604b13-bedb-4562-be52-7373c6352da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:03.357827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d604b13-bedb-4562-be52-7373c6352da5 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:03.358135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d604b13-bedb-4562-be52-7373c6352da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:03.358405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d604b13-bedb-4562-be52-7373c6352da5 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:03.359107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d604b13-bedb-4562-be52-7373c6352da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:03.359408 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d604b13-bedb-4562-be52-7373c6352da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:03.360898 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d604b13-bedb-4562-be52-7373c6352da5 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:03.373064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 533/1069] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:03.428592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5363cc-f02e-41ca-bd94-c56b16015a3e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:03.475100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fae7d8-978c-4059-a112-01231c864d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:03.476071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fae7d8-978c-4059-a112-01231c864d07 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:03.476567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fae7d8-978c-4059-a112-01231c864d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:03.477047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fae7d8-978c-4059-a112-01231c864d07 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:03.477948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fae7d8-978c-4059-a112-01231c864d07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:03.478428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fae7d8-978c-4059-a112-01231c864d07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:03.480032 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fae7d8-978c-4059-a112-01231c864d07 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:03.494126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 537/1070] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:52:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:03.500365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d604b13-bedb-4562-be52-7373c6352da5 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:03.613493 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d604b13-bedb-4562-be52-7373c6352da5 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:03.648762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51fd3d-cc7b-40d0-8cfc-0e94930bfd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:03.649542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51fd3d-cc7b-40d0-8cfc-0e94930bfd5d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:03.649760 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51fd3d-cc7b-40d0-8cfc-0e94930bfd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:03.650031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51fd3d-cc7b-40d0-8cfc-0e94930bfd5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:03.658183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51fd3d-cc7b-40d0-8cfc-0e94930bfd5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:03.659636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51fd3d-cc7b-40d0-8cfc-0e94930bfd5d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-3c51fd3d-cc7b-40d0-8cfc-0e94930bfd5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:52:03.671926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:03.673066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:03.674225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 534/1071] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Aug 1 11:52:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:03.680366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fae7d8-978c-4059-a112-01231c864d07 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:03.713764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:03.784908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fae7d8-978c-4059-a112-01231c864d07 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:03.820774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ec0ef-8448-4788-b5fb-ed4d426a9e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:03.821404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec0ef-8448-4788-b5fb-ed4d426a9e28 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:03.821683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec0ef-8448-4788-b5fb-ed4d426a9e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:03.821959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ec0ef-8448-4788-b5fb-ed4d426a9e28 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:03.822772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ec0ef-8448-4788-b5fb-ed4d426a9e28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:03.823077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ec0ef-8448-4788-b5fb-ed4d426a9e28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:03.824626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ec0ef-8448-4788-b5fb-ed4d426a9e28 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:03.842665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 538/1072] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Aug 1 11:52:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:03.848558 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51fd3d-cc7b-40d0-8cfc-0e94930bfd5d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:03.903329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:03.954557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51fd3d-cc7b-40d0-8cfc-0e94930bfd5d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:03.989853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b03d9e-51af-43de-81ce-ed9edb1e0c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:03.990498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b03d9e-51af-43de-81ce-ed9edb1e0c3b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:03.990781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b03d9e-51af-43de-81ce-ed9edb1e0c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:03.991053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b03d9e-51af-43de-81ce-ed9edb1e0c3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:04.002217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b03d9e-51af-43de-81ce-ed9edb1e0c3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:04.003681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b03d9e-51af-43de-81ce-ed9edb1e0c3b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-96b03d9e-51af-43de-81ce-ed9edb1e0c3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:52:04.018414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:04.019406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:04.020329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:52:04.020329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:52:04.020329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:52:04.020329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:04.024428 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 535/1073] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:52:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:04.025490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ec0ef-8448-4788-b5fb-ed4d426a9e28 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:04.127104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:04.137358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ec0ef-8448-4788-b5fb-ed4d426a9e28 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:04.174035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370aede0-4520-49c7-9eb2-7c49941011b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:04.174581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370aede0-4520-49c7-9eb2-7c49941011b0 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:04.174896 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370aede0-4520-49c7-9eb2-7c49941011b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:04.175167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370aede0-4520-49c7-9eb2-7c49941011b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:04.175873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370aede0-4520-49c7-9eb2-7c49941011b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:04.176177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370aede0-4520-49c7-9eb2-7c49941011b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:04.177600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370aede0-4520-49c7-9eb2-7c49941011b0 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:04.189759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 539/1074] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:04.195774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b03d9e-51af-43de-81ce-ed9edb1e0c3b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:04.309748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b03d9e-51af-43de-81ce-ed9edb1e0c3b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:04.311291 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:04.344637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67b308-70f4-47d6-bd07-cbb454fdc179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:04.345284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67b308-70f4-47d6-bd07-cbb454fdc179 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:04.345554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67b308-70f4-47d6-bd07-cbb454fdc179 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:04.345823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67b308-70f4-47d6-bd07-cbb454fdc179 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:04.352671 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67b308-70f4-47d6-bd07-cbb454fdc179 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:04.354122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e67b308-70f4-47d6-bd07-cbb454fdc179 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-0e67b308-70f4-47d6-bd07-cbb454fdc179 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:52:04.365899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:04.366897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:04.367931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:52:04.367931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:52:04.367931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:52:04.367931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:52:04.367931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:52:04.367931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:52:04.367931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:52:04.367931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:04.367931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 536/1075] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:52:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:04.373006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370aede0-4520-49c7-9eb2-7c49941011b0 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:04.487957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370aede0-4520-49c7-9eb2-7c49941011b0 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:04.525109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8ea99-c01b-4015-a97b-992532716569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:04.525754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8ea99-c01b-4015-a97b-992532716569 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:04.526002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8ea99-c01b-4015-a97b-992532716569 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:04.526295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8ea99-c01b-4015-a97b-992532716569 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:04.527030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8ea99-c01b-4015-a97b-992532716569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:04.527334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8ea99-c01b-4015-a97b-992532716569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:04.528810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8ea99-c01b-4015-a97b-992532716569 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:04.541889 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 540/1076] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:52:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:04.548223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67b308-70f4-47d6-bd07-cbb454fdc179 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:04.653464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67b308-70f4-47d6-bd07-cbb454fdc179 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:04.694099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf5657-03ed-410e-8fc7-5b7bc4aef97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:04.694788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf5657-03ed-410e-8fc7-5b7bc4aef97a None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:04.695067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf5657-03ed-410e-8fc7-5b7bc4aef97a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:04.695333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf5657-03ed-410e-8fc7-5b7bc4aef97a None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/db3ff1b21c784e8cb5bffcb79f230d08/roles/9a4d5a7314884a599e1b22288a1add9d` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:04.697616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf5657-03ed-410e-8fc7-5b7bc4aef97a None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=db3ff1b21c784e8cb5bffcb79f230d08, role_id=9a4d5a7314884a599e1b22288a1add9d)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:04.699079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf5657-03ed-410e-8fc7-5b7bc4aef97a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:04.713872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 537/1077] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:52:04 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/db3ff1b21c784e8cb5bffcb79f230d08/roles/9a4d5a7314884a599e1b22288a1add9d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:52:05.134436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180cb86-a52f-4378-a636-41a684a63efb None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:05.135025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180cb86-a52f-4378-a636-41a684a63efb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:05.135025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180cb86-a52f-4378-a636-41a684a63efb None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:05.136259 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 541/1078] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:05.141791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfdfa5-25ea-4e4e-83b0-27c08f75a97f None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:05.142150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfdfa5-25ea-4e4e-83b0-27c08f75a97f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:05.142427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfdfa5-25ea-4e4e-83b0-27c08f75a97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:05.162506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-36cfdfa5-25ea-4e4e-83b0-27c08f75a97f None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:05.163732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-36cfdfa5-25ea-4e4e-83b0-27c08f75a97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:05.182610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:05.240126 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 538/1079] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:05.262195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180cb86-a52f-4378-a636-41a684a63efb None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:05.275687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:05.370770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180cb86-a52f-4378-a636-41a684a63efb None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:05.407352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9163ae-8caf-477b-918d-65c66cad0f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:05.407924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9163ae-8caf-477b-918d-65c66cad0f6a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:05.408203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9163ae-8caf-477b-918d-65c66cad0f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:05.408502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9163ae-8caf-477b-918d-65c66cad0f6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:05.419174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9163ae-8caf-477b-918d-65c66cad0f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-4b9163ae-8caf-477b-918d-65c66cad0f6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:05.424256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:05.425329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:52:05.426290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 542/1080] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:05.431798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfdfa5-25ea-4e4e-83b0-27c08f75a97f None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:05.534406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfdfa5-25ea-4e4e-83b0-27c08f75a97f None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:05.569903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab1027-1864-4105-8105-f8bb8d0b5a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:05.570536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab1027-1864-4105-8105-f8bb8d0b5a87 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:05.570816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab1027-1864-4105-8105-f8bb8d0b5a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:05.571087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab1027-1864-4105-8105-f8bb8d0b5a87 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:05.571818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab1027-1864-4105-8105-f8bb8d0b5a87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:05.572121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab1027-1864-4105-8105-f8bb8d0b5a87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:05.573546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab1027-1864-4105-8105-f8bb8d0b5a87 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:05.582176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 539/1081] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:05.588253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9163ae-8caf-477b-918d-65c66cad0f6a None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:05.662198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:05.699045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9163ae-8caf-477b-918d-65c66cad0f6a None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:05.738484 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cc47f-1e33-4528-a806-ef28f3324f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:05.739150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cc47f-1e33-4528-a806-ef28f3324f33 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:05.739431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cc47f-1e33-4528-a806-ef28f3324f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:05.739691 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cc47f-1e33-4528-a806-ef28f3324f33 None admin] PATH_INFO: `/v3/users/dd3d609a83044b17ad19c08a0eb93045` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:05.741099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890cc47f-1e33-4528-a806-ef28f3324f33 None admin] RBAC: Authorizing `identity:get_user(user_id=dd3d609a83044b17ad19c08a0eb93045)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:05.742571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890cc47f-1e33-4528-a806-ef28f3324f33 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:05.744267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 543/1082] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:05 2022] GET /identity/v3/users/dd3d609a83044b17ad19c08a0eb93045 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:05.750403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab1027-1864-4105-8105-f8bb8d0b5a87 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:05.768666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:05.863155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab1027-1864-4105-8105-f8bb8d0b5a87 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:05.897102 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc8b7f-7311-434b-9556-de33f266c923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:05.897750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc8b7f-7311-434b-9556-de33f266c923 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:05.898026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc8b7f-7311-434b-9556-de33f266c923 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:05.898296 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc8b7f-7311-434b-9556-de33f266c923 None admin] PATH_INFO: `/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:05.899474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc8b7f-7311-434b-9556-de33f266c923 None admin] RBAC: Authorizing `identity:get_project(project_id=1c61c0f1dd8b47969fad7c1f653bf27c)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:05.900979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc8b7f-7311-434b-9556-de33f266c923 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:05.902652 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 540/1083] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:05 2022] GET /identity/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:05.909130 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cc47f-1e33-4528-a806-ef28f3324f33 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:06.008316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cc47f-1e33-4528-a806-ef28f3324f33 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:06.041545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8847f4-164f-465d-88da-10f6fc52da0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:06.042219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8847f4-164f-465d-88da-10f6fc52da0f None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:06.042501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8847f4-164f-465d-88da-10f6fc52da0f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:06.042772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8847f4-164f-465d-88da-10f6fc52da0f None admin] PATH_INFO: `/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c/users/dd3d609a83044b17ad19c08a0eb93045/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:06.044764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8847f4-164f-465d-88da-10f6fc52da0f None admin] RBAC: Authorizing `identity:create_grant(project_id=1c61c0f1dd8b47969fad7c1f653bf27c, user_id=dd3d609a83044b17ad19c08a0eb93045, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:06.046195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8847f4-164f-465d-88da-10f6fc52da0f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:06.055567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:06.061780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 544/1084] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:52:05 2022] PUT /identity/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c/users/dd3d609a83044b17ad19c08a0eb93045/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:52:06.702010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b8ce-0699-482b-ac70-1f6335939014 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:06.702245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b8ce-0699-482b-ac70-1f6335939014 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:06.702554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83b8ce-0699-482b-ac70-1f6335939014 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:06.703992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 541/1085] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:06.709426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4e1d2-68cf-4c61-a9e7-1d80a2509537 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:06.709899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4e1d2-68cf-4c61-a9e7-1d80a2509537 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:06.709899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4e1d2-68cf-4c61-a9e7-1d80a2509537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:06.729023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-73d4e1d2-68cf-4c61-a9e7-1d80a2509537 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:06.730374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-73d4e1d2-68cf-4c61-a9e7-1d80a2509537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:06.799806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 545/1086] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:06.823130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83b8ce-0699-482b-ac70-1f6335939014 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:06.914865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:06.934634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83b8ce-0699-482b-ac70-1f6335939014 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:06.968587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd537d8d-418e-4527-88af-2cc752427bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:06.969189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd537d8d-418e-4527-88af-2cc752427bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:06.969470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd537d8d-418e-4527-88af-2cc752427bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:06.969740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd537d8d-418e-4527-88af-2cc752427bd4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:06.980325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd537d8d-418e-4527-88af-2cc752427bd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-fd537d8d-418e-4527-88af-2cc752427bd4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:06.985637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:06.986566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 01 11:52:06.987394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:06.988520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 542/1087] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Aug 1 11:52:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:06.992973 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4e1d2-68cf-4c61-a9e7-1d80a2509537 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:07.097476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4e1d2-68cf-4c61-a9e7-1d80a2509537 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:07.124622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:07.133791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1da33-3e3a-40d7-8602-ff1c211fe62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:07.134426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1da33-3e3a-40d7-8602-ff1c211fe62a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:07.134705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1da33-3e3a-40d7-8602-ff1c211fe62a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:07.134978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e1da33-3e3a-40d7-8602-ff1c211fe62a None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:07.135710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1da33-3e3a-40d7-8602-ff1c211fe62a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:07.136008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1da33-3e3a-40d7-8602-ff1c211fe62a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:07.137439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e1da33-3e3a-40d7-8602-ff1c211fe62a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:07.146186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 546/1088] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Aug 1 11:52:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:07.152280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd537d8d-418e-4527-88af-2cc752427bd4 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:07.258131 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd537d8d-418e-4527-88af-2cc752427bd4 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:07.293161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b89b53-5510-40d2-818c-5ee15d287f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:07.293807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b89b53-5510-40d2-818c-5ee15d287f82 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:07.294083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b89b53-5510-40d2-818c-5ee15d287f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:07.294353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b89b53-5510-40d2-818c-5ee15d287f82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:07.304738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b89b53-5510-40d2-818c-5ee15d287f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:07.306180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b89b53-5510-40d2-818c-5ee15d287f82 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:07.307900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-d5b89b53-5510-40d2-818c-5ee15d287f82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:52:07.321463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:07.322399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:07.323304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:52:07.323304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:52:07.323304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:52:07.323304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:07.323304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 543/1089] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:52:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:07.328536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1da33-3e3a-40d7-8602-ff1c211fe62a None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:07.428750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e1da33-3e3a-40d7-8602-ff1c211fe62a None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:07.461913 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e56b4b-c51c-4dea-96eb-7d59e86986b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:07.462538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e56b4b-c51c-4dea-96eb-7d59e86986b5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:07.462815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e56b4b-c51c-4dea-96eb-7d59e86986b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:07.463086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e56b4b-c51c-4dea-96eb-7d59e86986b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:07.463784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e56b4b-c51c-4dea-96eb-7d59e86986b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:07.464072 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e56b4b-c51c-4dea-96eb-7d59e86986b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:07.465494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e56b4b-c51c-4dea-96eb-7d59e86986b5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:07.477621 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 547/1090] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:52:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:07.483579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b89b53-5510-40d2-818c-5ee15d287f82 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:07.591059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b89b53-5510-40d2-818c-5ee15d287f82 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:07.626848 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ae7ff-0779-40ca-99ec-2fd4056c496a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:07.627494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ae7ff-0779-40ca-99ec-2fd4056c496a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:07.627770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ae7ff-0779-40ca-99ec-2fd4056c496a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:07.628039 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ae7ff-0779-40ca-99ec-2fd4056c496a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:07.635846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ae7ff-0779-40ca-99ec-2fd4056c496a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:07.637325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ae7ff-0779-40ca-99ec-2fd4056c496a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-999ae7ff-0779-40ca-99ec-2fd4056c496a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:52:07.649075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:07.650047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:07.650986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 544/1091] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Aug 1 11:52:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:07.656217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e56b4b-c51c-4dea-96eb-7d59e86986b5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:07.706673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:07.757667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e56b4b-c51c-4dea-96eb-7d59e86986b5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:07.798228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04fcea4-4374-490c-9482-598940eaadd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:07.798864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fcea4-4374-490c-9482-598940eaadd4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:07.799138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fcea4-4374-490c-9482-598940eaadd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:07.799406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04fcea4-4374-490c-9482-598940eaadd4 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:07.800217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fcea4-4374-490c-9482-598940eaadd4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:07.800551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fcea4-4374-490c-9482-598940eaadd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:07.801999 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04fcea4-4374-490c-9482-598940eaadd4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:07.819470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 548/1092] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Aug 1 11:52:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:07.825467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ae7ff-0779-40ca-99ec-2fd4056c496a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:07.888521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:07.935318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ae7ff-0779-40ca-99ec-2fd4056c496a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:07.968388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99424ada-87a2-495d-8034-a56dab321d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:07.969014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99424ada-87a2-495d-8034-a56dab321d38 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:07.969391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99424ada-87a2-495d-8034-a56dab321d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:07.969661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99424ada-87a2-495d-8034-a56dab321d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.038860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99424ada-87a2-495d-8034-a56dab321d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:08.039376 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbafcae-5520-40ec-b24a-da770c5c2a2b None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.039658 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbafcae-5520-40ec-b24a-da770c5c2a2b None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.039926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbafcae-5520-40ec-b24a-da770c5c2a2b None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.040369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99424ada-87a2-495d-8034-a56dab321d38 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:08.041332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 549/1093] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:52:08.046451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf968176-02b9-4723-8a76-43486fb403e1 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.046814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf968176-02b9-4723-8a76-43486fb403e1 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.047192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf968176-02b9-4723-8a76-43486fb403e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.065752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-bf968176-02b9-4723-8a76-43486fb403e1 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:08.066960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bf968176-02b9-4723-8a76-43486fb403e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:08.097444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:08.137452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 550/1094] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:52:08.143035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 545/1095] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:08.148743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf968176-02b9-4723-8a76-43486fb403e1 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:08.160552 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99424ada-87a2-495d-8034-a56dab321d38 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:08.187743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:08.249390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf968176-02b9-4723-8a76-43486fb403e1 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:08.269061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99424ada-87a2-495d-8034-a56dab321d38 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:08.285830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b027d32-93a1-4b35-a749-b78e1e3201e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:08.286472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b027d32-93a1-4b35-a749-b78e1e3201e5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.286746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b027d32-93a1-4b35-a749-b78e1e3201e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.287013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b027d32-93a1-4b35-a749-b78e1e3201e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.296966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b027d32-93a1-4b35-a749-b78e1e3201e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:08.298481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b027d32-93a1-4b35-a749-b78e1e3201e5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:08.302953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23788dcd-05bf-4164-9537-24f627bf3d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:08.303600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23788dcd-05bf-4164-9537-24f627bf3d3d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.303880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23788dcd-05bf-4164-9537-24f627bf3d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.304150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23788dcd-05bf-4164-9537-24f627bf3d3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-1b027d32-93a1-4b35-a749-b78e1e3201e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:52:08.313869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:08.314817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:08.315969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:52:08.315969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:52:08.315969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:52:08.315969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:52:08.316441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23788dcd-05bf-4164-9537-24f627bf3d3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:08.316515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 551/1096] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:52:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-23788dcd-05bf-4164-9537-24f627bf3d3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:08.319526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:08.320497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:08.321348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:08.322633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b027d32-93a1-4b35-a749-b78e1e3201e5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:08.322719 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 546/1097] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:52:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:52:08.326525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23788dcd-05bf-4164-9537-24f627bf3d3d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:08.342632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:08.411527 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:08.422219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b027d32-93a1-4b35-a749-b78e1e3201e5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:08.426888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23788dcd-05bf-4164-9537-24f627bf3d3d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:08.455838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fd92e-5339-4d2b-99de-bbd139474a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:08.456496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fd92e-5339-4d2b-99de-bbd139474a65 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.456774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fd92e-5339-4d2b-99de-bbd139474a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.457046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fd92e-5339-4d2b-99de-bbd139474a65 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.457743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fd92e-5339-4d2b-99de-bbd139474a65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:08.458050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fd92e-5339-4d2b-99de-bbd139474a65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:08.459466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fd92e-5339-4d2b-99de-bbd139474a65 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:08.460300 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e7338-fd0c-4c8d-bb19-edb54c4744e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:08.460955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e7338-fd0c-4c8d-bb19-edb54c4744e7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.461234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e7338-fd0c-4c8d-bb19-edb54c4744e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.461502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e7338-fd0c-4c8d-bb19-edb54c4744e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.462232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e7338-fd0c-4c8d-bb19-edb54c4744e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:08.462537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e7338-fd0c-4c8d-bb19-edb54c4744e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:08.463896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e7338-fd0c-4c8d-bb19-edb54c4744e7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:08.471229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 552/1098] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:08.472176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 547/1099] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:52:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:08.477575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fd92e-5339-4d2b-99de-bbd139474a65 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:08.477800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e7338-fd0c-4c8d-bb19-edb54c4744e7 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:08.554708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:08.577874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e7338-fd0c-4c8d-bb19-edb54c4744e7 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:08.581442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fd92e-5339-4d2b-99de-bbd139474a65 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:08.614541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219259a8-cd97-45c6-a962-dff394a6fb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:08.615340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219259a8-cd97-45c6-a962-dff394a6fb6d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.615689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219259a8-cd97-45c6-a962-dff394a6fb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.616033 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219259a8-cd97-45c6-a962-dff394a6fb6d None admin] PATH_INFO: `/v3/users/dd3d609a83044b17ad19c08a0eb93045` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.617351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219259a8-cd97-45c6-a962-dff394a6fb6d None admin] RBAC: Authorizing `identity:get_user(user_id=dd3d609a83044b17ad19c08a0eb93045)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:08.618804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219259a8-cd97-45c6-a962-dff394a6fb6d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:08.618943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d65c9-4998-45ff-8174-b7a514b9e724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:08.619593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d65c9-4998-45ff-8174-b7a514b9e724 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.619873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d65c9-4998-45ff-8174-b7a514b9e724 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.620145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d65c9-4998-45ff-8174-b7a514b9e724 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.620473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 548/1100] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:08 2022] GET /identity/v3/users/dd3d609a83044b17ad19c08a0eb93045 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:08.626350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219259a8-cd97-45c6-a962-dff394a6fb6d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:08.626995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d65c9-4998-45ff-8174-b7a514b9e724 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:08.628485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d65c9-4998-45ff-8174-b7a514b9e724 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-f72d65c9-4998-45ff-8174-b7a514b9e724 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:52:08.639761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:08.640722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:08.641671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:52:08.641671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:52:08.641671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:52:08.641671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:52:08.641671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:52:08.641671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:52:08.641671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:52:08.641671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:52:08.641671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 553/1101] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:52:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:08.646808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d65c9-4998-45ff-8174-b7a514b9e724 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:08.659037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:08.729435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219259a8-cd97-45c6-a962-dff394a6fb6d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:08.747911 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d65c9-4998-45ff-8174-b7a514b9e724 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:08.764171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc035dd8-2884-42fa-9804-798ea8faf248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:08.764828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc035dd8-2884-42fa-9804-798ea8faf248 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.765108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc035dd8-2884-42fa-9804-798ea8faf248 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.765380 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc035dd8-2884-42fa-9804-798ea8faf248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.779724 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:08.781560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1e55c-5cb3-4dfc-948b-9164a426be71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:08.782214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1e55c-5cb3-4dfc-948b-9164a426be71 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.782608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1e55c-5cb3-4dfc-948b-9164a426be71 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.782888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e1e55c-5cb3-4dfc-948b-9164a426be71 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.783649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1e55c-5cb3-4dfc-948b-9164a426be71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:08.783948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1e55c-5cb3-4dfc-948b-9164a426be71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:08.785436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e1e55c-5cb3-4dfc-948b-9164a426be71 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:08.797915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 554/1102] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:52:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:08.804095 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1e55c-5cb3-4dfc-948b-9164a426be71 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:08.838356 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc035dd8-2884-42fa-9804-798ea8faf248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:08.839831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc035dd8-2884-42fa-9804-798ea8faf248 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:08.880129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:08.914885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e1e55c-5cb3-4dfc-948b-9164a426be71 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:08.942843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 549/1103] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:08.948896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc035dd8-2884-42fa-9804-798ea8faf248 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:08.950737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b0d15-75b9-4372-91ab-7c11b6707dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:08.951365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b0d15-75b9-4372-91ab-7c11b6707dab None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:08.951644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b0d15-75b9-4372-91ab-7c11b6707dab None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:08.951919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b0d15-75b9-4372-91ab-7c11b6707dab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:08.952746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b0d15-75b9-4372-91ab-7c11b6707dab None admin] RBAC: Adding query filter params (role.id=9a4d5a7314884a599e1b22288a1add9d, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=db3ff1b21c784e8cb5bffcb79f230d08) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:08.953058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b0d15-75b9-4372-91ab-7c11b6707dab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:08.954528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b0d15-75b9-4372-91ab-7c11b6707dab None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:08.961374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 555/1104] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:52:08 2022] GET /identity/v3/role_assignments?user.id=db3ff1b21c784e8cb5bffcb79f230d08&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=9a4d5a7314884a599e1b22288a1add9d => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:09.000450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:09.060069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc035dd8-2884-42fa-9804-798ea8faf248 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:09.099359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d73a02-447b-4696-93a1-49c9d7779ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:09.100038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d73a02-447b-4696-93a1-49c9d7779ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:09.100317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d73a02-447b-4696-93a1-49c9d7779ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:09.100618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d73a02-447b-4696-93a1-49c9d7779ff7 None admin] PATH_INFO: `/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:09.101883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d73a02-447b-4696-93a1-49c9d7779ff7 None admin] RBAC: Authorizing `identity:get_project(project_id=1c61c0f1dd8b47969fad7c1f653bf27c)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:09.103346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d73a02-447b-4696-93a1-49c9d7779ff7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:09.105142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 550/1105] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:08 2022] GET /identity/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:09.112123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b0d15-75b9-4372-91ab-7c11b6707dab None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:09.219738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b0d15-75b9-4372-91ab-7c11b6707dab None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:09.255516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800cda8-65e9-4947-85be-0ea9969ee284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:09.256066 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800cda8-65e9-4947-85be-0ea9969ee284 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:09.256341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800cda8-65e9-4947-85be-0ea9969ee284 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:09.256640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800cda8-65e9-4947-85be-0ea9969ee284 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:09.257439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800cda8-65e9-4947-85be-0ea9969ee284 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=1c61c0f1dd8b47969fad7c1f653bf27c, user.id=dd3d609a83044b17ad19c08a0eb93045) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:09.257746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800cda8-65e9-4947-85be-0ea9969ee284 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:09.259197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800cda8-65e9-4947-85be-0ea9969ee284 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:09.261103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:09.266920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 556/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:52:09 2022] GET /identity/v3/role_assignments?user.id=dd3d609a83044b17ad19c08a0eb93045&scope.project.id=1c61c0f1dd8b47969fad7c1f653bf27c&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:11.309472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d64d3-8510-4096-9572-edbed4cdca34 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:11.325265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d64d3-8510-4096-9572-edbed4cdca34 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:11.325265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d64d3-8510-4096-9572-edbed4cdca34 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:11.325265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 551/1107] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:11.325735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9807-75a6-4308-aa57-e622c9c3d21b None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:11.325735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9807-75a6-4308-aa57-e622c9c3d21b None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:11.325735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9807-75a6-4308-aa57-e622c9c3d21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:11.345546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-578d9807-75a6-4308-aa57-e622c9c3d21b None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:11.346775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-578d9807-75a6-4308-aa57-e622c9c3d21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:11.430011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 557/1108] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:11.466681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d64d3-8510-4096-9572-edbed4cdca34 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:11.589853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d64d3-8510-4096-9572-edbed4cdca34 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:11.603356 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:11.635204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768479f7-9465-43d0-b773-0095e0883152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:11.635855 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768479f7-9465-43d0-b773-0095e0883152 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:11.636130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768479f7-9465-43d0-b773-0095e0883152 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:11.636436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768479f7-9465-43d0-b773-0095e0883152 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:11.637715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768479f7-9465-43d0-b773-0095e0883152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:11.639168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768479f7-9465-43d0-b773-0095e0883152 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:11.640833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 552/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:52:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:11.647115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9807-75a6-4308-aa57-e622c9c3d21b None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:11.765078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9807-75a6-4308-aa57-e622c9c3d21b None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:11.796582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:11.805455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61321ea-2d71-414d-a839-7e82a17fd493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:11.806080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61321ea-2d71-414d-a839-7e82a17fd493 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:11.806359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61321ea-2d71-414d-a839-7e82a17fd493 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:11.806632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61321ea-2d71-414d-a839-7e82a17fd493 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:11.807326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61321ea-2d71-414d-a839-7e82a17fd493 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:11.808812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61321ea-2d71-414d-a839-7e82a17fd493 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:11.832377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 558/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:52:11 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:52:13.845266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b849d-6a4d-4092-a837-161e5cb834b6 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:13.845851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b849d-6a4d-4092-a837-161e5cb834b6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:13.845851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b849d-6a4d-4092-a837-161e5cb834b6 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:13.847190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 553/1111] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:13.855097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0365e-fe67-48f2-b3dd-fd1baf74334c None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:13.855632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0365e-fe67-48f2-b3dd-fd1baf74334c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:13.855632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0365e-fe67-48f2-b3dd-fd1baf74334c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:13.881852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-41a0365e-fe67-48f2-b3dd-fd1baf74334c None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:13.883059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-41a0365e-fe67-48f2-b3dd-fd1baf74334c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:13.961975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 559/1112] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:13.987392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b849d-6a4d-4092-a837-161e5cb834b6 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:14.108840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b849d-6a4d-4092-a837-161e5cb834b6 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:14.151174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7ccbc-92b3-43e3-bf9a-6edb9b365bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:14.151829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7ccbc-92b3-43e3-bf9a-6edb9b365bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:14.152110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7ccbc-92b3-43e3-bf9a-6edb9b365bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:14.152412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7ccbc-92b3-43e3-bf9a-6edb9b365bf6 None admin] PATH_INFO: `/v3/roles/8e13cd96bb124d52b5c8883fe960424f` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:14.153907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7ccbc-92b3-43e3-bf9a-6edb9b365bf6 None admin] RBAC: Authorizing `identity:get_role(role_id=8e13cd96bb124d52b5c8883fe960424f)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:14.155299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7ccbc-92b3-43e3-bf9a-6edb9b365bf6 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:14.156755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 554/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:13 2022] GET /identity/v3/roles/8e13cd96bb124d52b5c8883fe960424f => generated 223 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:14.163345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0365e-fe67-48f2-b3dd-fd1baf74334c None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:14.270900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0365e-fe67-48f2-b3dd-fd1baf74334c None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:14.288110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:14.311423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752978a-6496-494b-9da5-267c06b5cde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:14.312080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752978a-6496-494b-9da5-267c06b5cde7 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:14.312388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752978a-6496-494b-9da5-267c06b5cde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:14.312654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752978a-6496-494b-9da5-267c06b5cde7 None admin] PATH_INFO: `/v3/users/92c87213b77f4b90b8fd74ad582b294c` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:14.326418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752978a-6496-494b-9da5-267c06b5cde7 None admin] RBAC: Authorizing `identity:get_user(user_id=92c87213b77f4b90b8fd74ad582b294c)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:14.327872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752978a-6496-494b-9da5-267c06b5cde7 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:14.329690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 560/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:14 2022] GET /identity/v3/users/92c87213b77f4b90b8fd74ad582b294c => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:14.335801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7ccbc-92b3-43e3-bf9a-6edb9b365bf6 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:14.391868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:14.450635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7ccbc-92b3-43e3-bf9a-6edb9b365bf6 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:14.486505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444fa2-b5cb-4c62-8946-5b7ccb25d325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:14.487138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444fa2-b5cb-4c62-8946-5b7ccb25d325 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:14.487419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444fa2-b5cb-4c62-8946-5b7ccb25d325 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:14.487692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48444fa2-b5cb-4c62-8946-5b7ccb25d325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:14.564370 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48444fa2-b5cb-4c62-8946-5b7ccb25d325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:14.565830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48444fa2-b5cb-4c62-8946-5b7ccb25d325 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:14.614621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:14.670034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 555/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:14.676315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752978a-6496-494b-9da5-267c06b5cde7 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:14.780562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752978a-6496-494b-9da5-267c06b5cde7 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:14.817540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d6847-d60f-467c-aa00-14fdd266c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:14.818176 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d6847-d60f-467c-aa00-14fdd266c66a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:14.818451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d6847-d60f-467c-aa00-14fdd266c66a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:14.818720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d6847-d60f-467c-aa00-14fdd266c66a None admin] PATH_INFO: `/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:14.819915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d6847-d60f-467c-aa00-14fdd266c66a None admin] RBAC: Authorizing `identity:get_project(project_id=1c61c0f1dd8b47969fad7c1f653bf27c)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:14.821388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d6847-d60f-467c-aa00-14fdd266c66a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:14.823062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 561/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:14 2022] GET /identity/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:14.829580 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444fa2-b5cb-4c62-8946-5b7ccb25d325 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:14.946697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48444fa2-b5cb-4c62-8946-5b7ccb25d325 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:14.987205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e34010-5ea8-4085-a5fc-de0948de998f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:14.987820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e34010-5ea8-4085-a5fc-de0948de998f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:14.988099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e34010-5ea8-4085-a5fc-de0948de998f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:14.988397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e34010-5ea8-4085-a5fc-de0948de998f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:14.989198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e34010-5ea8-4085-a5fc-de0948de998f None admin] RBAC: Adding query filter params (role.id=8e13cd96bb124d52b5c8883fe960424f, scope.project.id=1c61c0f1dd8b47969fad7c1f653bf27c, user.id=92c87213b77f4b90b8fd74ad582b294c) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:14.989504 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e34010-5ea8-4085-a5fc-de0948de998f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:14.990944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e34010-5ea8-4085-a5fc-de0948de998f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:14.996402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 556/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:52:14 2022] GET /identity/v3/role_assignments?user.id=92c87213b77f4b90b8fd74ad582b294c&scope.project.id=1c61c0f1dd8b47969fad7c1f653bf27c&role.id=8e13cd96bb124d52b5c8883fe960424f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:16.979355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3fa0-5854-45ae-9e5f-9a5edf3a9bf1 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:16.979970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3fa0-5854-45ae-9e5f-9a5edf3a9bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:16.979970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3fa0-5854-45ae-9e5f-9a5edf3a9bf1 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:16.981273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 562/1118] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:16.987011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250e363-6f1c-4513-bbbf-2f74c07774aa None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:16.987431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250e363-6f1c-4513-bbbf-2f74c07774aa None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:16.987561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250e363-6f1c-4513-bbbf-2f74c07774aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:17.006354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d250e363-6f1c-4513-bbbf-2f74c07774aa None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:17.007575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d250e363-6f1c-4513-bbbf-2f74c07774aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:17.014917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:17.084940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 557/1119] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:17.108839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3fa0-5854-45ae-9e5f-9a5edf3a9bf1 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:17.157518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:17.227655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3fa0-5854-45ae-9e5f-9a5edf3a9bf1 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:17.266712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eafbe-d11a-4096-8ff9-2e45ec5ad4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:17.267365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eafbe-d11a-4096-8ff9-2e45ec5ad4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:17.267750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eafbe-d11a-4096-8ff9-2e45ec5ad4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:17.267932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13eafbe-d11a-4096-8ff9-2e45ec5ad4e3 None admin] PATH_INFO: `/v3/roles/8e13cd96bb124d52b5c8883fe960424f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:17.269442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13eafbe-d11a-4096-8ff9-2e45ec5ad4e3 None admin] RBAC: Authorizing `identity:get_role(role_id=8e13cd96bb124d52b5c8883fe960424f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:17.270825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13eafbe-d11a-4096-8ff9-2e45ec5ad4e3 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:17.272221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 563/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:17 2022] GET /identity/v3/roles/8e13cd96bb124d52b5c8883fe960424f => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:17.279120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250e363-6f1c-4513-bbbf-2f74c07774aa None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:17.382488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250e363-6f1c-4513-bbbf-2f74c07774aa None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:17.420309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6b633-3027-4eb8-aba0-390a387f9782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:17.420989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6b633-3027-4eb8-aba0-390a387f9782 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:17.421313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6b633-3027-4eb8-aba0-390a387f9782 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:17.421653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6b633-3027-4eb8-aba0-390a387f9782 None admin] PATH_INFO: `/v3/users/92c87213b77f4b90b8fd74ad582b294c` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:17.423063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6b633-3027-4eb8-aba0-390a387f9782 None admin] RBAC: Authorizing `identity:get_user(user_id=92c87213b77f4b90b8fd74ad582b294c)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:17.424583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6b633-3027-4eb8-aba0-390a387f9782 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:17.426183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 558/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:17 2022] GET /identity/v3/users/92c87213b77f4b90b8fd74ad582b294c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:17.432324 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eafbe-d11a-4096-8ff9-2e45ec5ad4e3 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:17.534336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13eafbe-d11a-4096-8ff9-2e45ec5ad4e3 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:17.543670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:17.569929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986b569-6f93-405c-b349-a6006f175e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:17.570571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986b569-6f93-405c-b349-a6006f175e93 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:17.570848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986b569-6f93-405c-b349-a6006f175e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:17.571118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986b569-6f93-405c-b349-a6006f175e93 None admin] PATH_INFO: `/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:17.572283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986b569-6f93-405c-b349-a6006f175e93 None admin] RBAC: Authorizing `identity:get_project(project_id=1c61c0f1dd8b47969fad7c1f653bf27c)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:17.573752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986b569-6f93-405c-b349-a6006f175e93 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:17.575439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 564/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:17 2022] GET /identity/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:17.582089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6b633-3027-4eb8-aba0-390a387f9782 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:17.594944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:17.686265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6b633-3027-4eb8-aba0-390a387f9782 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:17.723036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d439224-e6a2-4d4f-9f89-90db5384235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:17.723724 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d439224-e6a2-4d4f-9f89-90db5384235e None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:17.724006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d439224-e6a2-4d4f-9f89-90db5384235e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:17.724286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d439224-e6a2-4d4f-9f89-90db5384235e None admin] PATH_INFO: `/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c/users/92c87213b77f4b90b8fd74ad582b294c/roles/8e13cd96bb124d52b5c8883fe960424f` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:17.726669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d439224-e6a2-4d4f-9f89-90db5384235e None admin] RBAC: Authorizing `identity:create_grant(project_id=1c61c0f1dd8b47969fad7c1f653bf27c, user_id=92c87213b77f4b90b8fd74ad582b294c, role_id=8e13cd96bb124d52b5c8883fe960424f)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:17.728128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d439224-e6a2-4d4f-9f89-90db5384235e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:17.742893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 559/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:52:17 2022] PUT /identity/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c/users/92c87213b77f4b90b8fd74ad582b294c/roles/8e13cd96bb124d52b5c8883fe960424f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:52:19.695384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eecc2-d333-49d9-aa41-f6fca179f916 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:19.695949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eecc2-d333-49d9-aa41-f6fca179f916 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:19.695949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eecc2-d333-49d9-aa41-f6fca179f916 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:19.697323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 565/1124] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:19.703830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d92ed7-da0c-49c0-bbfb-7e4e8a9354c3 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:19.704352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d92ed7-da0c-49c0-bbfb-7e4e8a9354c3 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:19.704579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d92ed7-da0c-49c0-bbfb-7e4e8a9354c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:19.726275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-09d92ed7-da0c-49c0-bbfb-7e4e8a9354c3 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:19.727512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-09d92ed7-da0c-49c0-bbfb-7e4e8a9354c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:19.794581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:19.800470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 560/1125] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:19.822833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5eecc2-d333-49d9-aa41-f6fca179f916 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:19.931740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5eecc2-d333-49d9-aa41-f6fca179f916 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:19.978791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cf90-cdbc-44ea-a902-641a1bcaaa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:19.979363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cf90-cdbc-44ea-a902-641a1bcaaa04 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:19.979644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cf90-cdbc-44ea-a902-641a1bcaaa04 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:19.979918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cf90-cdbc-44ea-a902-641a1bcaaa04 None admin] PATH_INFO: `/v3/roles/8e13cd96bb124d52b5c8883fe960424f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:19.981434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813cf90-cdbc-44ea-a902-641a1bcaaa04 None admin] RBAC: Authorizing `identity:get_role(role_id=8e13cd96bb124d52b5c8883fe960424f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:19.983201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813cf90-cdbc-44ea-a902-641a1bcaaa04 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:19.984605 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 566/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:19 2022] GET /identity/v3/roles/8e13cd96bb124d52b5c8883fe960424f => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:19.990880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d92ed7-da0c-49c0-bbfb-7e4e8a9354c3 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:20.096793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d92ed7-da0c-49c0-bbfb-7e4e8a9354c3 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:20.134188 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be440476-7d1b-43ce-b4ce-4949132441e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:20.134840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be440476-7d1b-43ce-b4ce-4949132441e7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:20.135119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be440476-7d1b-43ce-b4ce-4949132441e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:20.135394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be440476-7d1b-43ce-b4ce-4949132441e7 None admin] PATH_INFO: `/v3/users/92c87213b77f4b90b8fd74ad582b294c` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:20.136620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be440476-7d1b-43ce-b4ce-4949132441e7 None admin] RBAC: Authorizing `identity:get_user(user_id=92c87213b77f4b90b8fd74ad582b294c)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:20.138082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be440476-7d1b-43ce-b4ce-4949132441e7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:20.139759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 561/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:19 2022] GET /identity/v3/users/92c87213b77f4b90b8fd74ad582b294c => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:20.145959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cf90-cdbc-44ea-a902-641a1bcaaa04 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:20.195351 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:20.252800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cf90-cdbc-44ea-a902-641a1bcaaa04 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:20.292900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d924f-2bcc-4b5e-8827-a30e99475d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:20.293548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d924f-2bcc-4b5e-8827-a30e99475d0b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:20.293831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d924f-2bcc-4b5e-8827-a30e99475d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:20.294103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d924f-2bcc-4b5e-8827-a30e99475d0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:20.363864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d924f-2bcc-4b5e-8827-a30e99475d0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:20.365368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68d924f-2bcc-4b5e-8827-a30e99475d0b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:20.410386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:20.467599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 567/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:20.473902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be440476-7d1b-43ce-b4ce-4949132441e7 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:20.543828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:20.578902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be440476-7d1b-43ce-b4ce-4949132441e7 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:20.616037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927a31d-d6ec-44a5-864e-b765459e454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:20.616716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927a31d-d6ec-44a5-864e-b765459e454a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:20.616996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927a31d-d6ec-44a5-864e-b765459e454a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:20.617253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927a31d-d6ec-44a5-864e-b765459e454a None admin] PATH_INFO: `/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:20.618472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927a31d-d6ec-44a5-864e-b765459e454a None admin] RBAC: Authorizing `identity:get_project(project_id=1c61c0f1dd8b47969fad7c1f653bf27c)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:20.619931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927a31d-d6ec-44a5-864e-b765459e454a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:20.621670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 562/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:20 2022] GET /identity/v3/projects/1c61c0f1dd8b47969fad7c1f653bf27c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:20.628198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d924f-2bcc-4b5e-8827-a30e99475d0b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:20.733809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d924f-2bcc-4b5e-8827-a30e99475d0b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:20.769630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b5d50-4580-4b2c-b76d-4118fa3a5fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:20.770327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b5d50-4580-4b2c-b76d-4118fa3a5fde None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:20.770662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b5d50-4580-4b2c-b76d-4118fa3a5fde None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:20.770985 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b5d50-4580-4b2c-b76d-4118fa3a5fde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:20.771828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b5d50-4580-4b2c-b76d-4118fa3a5fde None admin] RBAC: Adding query filter params (role.id=8e13cd96bb124d52b5c8883fe960424f, scope.project.id=1c61c0f1dd8b47969fad7c1f653bf27c, user.id=92c87213b77f4b90b8fd74ad582b294c) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:20.772193 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b5d50-4580-4b2c-b76d-4118fa3a5fde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:20.773762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b5d50-4580-4b2c-b76d-4118fa3a5fde None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:20.779935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 568/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:52:20 2022] GET /identity/v3/role_assignments?user.id=92c87213b77f4b90b8fd74ad582b294c&scope.project.id=1c61c0f1dd8b47969fad7c1f653bf27c&role.id=8e13cd96bb124d52b5c8883fe960424f => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:22.770556 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541439e-0293-40c6-af1a-e319c614948f None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:22.771141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541439e-0293-40c6-af1a-e319c614948f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:22.771141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541439e-0293-40c6-af1a-e319c614948f None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:22.772466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 563/1131] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:52:22.778062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026a439-5ad4-42f9-afac-f01bcc559c09 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:22.778524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026a439-5ad4-42f9-afac-f01bcc559c09 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:22.778633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026a439-5ad4-42f9-afac-f01bcc559c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:22.798847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3026a439-5ad4-42f9-afac-f01bcc559c09 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:22.800071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3026a439-5ad4-42f9-afac-f01bcc559c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:22.802207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:22.872465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 569/1132] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:22.895836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541439e-0293-40c6-af1a-e319c614948f None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:23.003705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541439e-0293-40c6-af1a-e319c614948f None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:23.038914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a13c50-f791-4b56-ad35-d751c9b95742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:23.039475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13c50-f791-4b56-ad35-d751c9b95742 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:23.039752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13c50-f791-4b56-ad35-d751c9b95742 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:23.040024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a13c50-f791-4b56-ad35-d751c9b95742 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:23.041236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a13c50-f791-4b56-ad35-d751c9b95742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:23.042688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a13c50-f791-4b56-ad35-d751c9b95742 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:23.044315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 564/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:52:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:23.050766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026a439-5ad4-42f9-afac-f01bcc559c09 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:23.152928 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026a439-5ad4-42f9-afac-f01bcc559c09 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:23.187513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed5e17-5150-4284-a948-980c4c65bdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:23.188184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed5e17-5150-4284-a948-980c4c65bdd9 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:23.188481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed5e17-5150-4284-a948-980c4c65bdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:23.188752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed5e17-5150-4284-a948-980c4c65bdd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:23.189432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed5e17-5150-4284-a948-980c4c65bdd9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:23.190916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed5e17-5150-4284-a948-980c4c65bdd9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:23.200399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:23.218709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 570/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:52:23 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:52:25.248727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc3653-8a1d-4543-9e2a-44e639d82920 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:25.249271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc3653-8a1d-4543-9e2a-44e639d82920 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:25.249271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc3653-8a1d-4543-9e2a-44e639d82920 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:25.250972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 565/1135] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:25.257142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130ec2e-6bad-481f-8afa-e2e8721b3309 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:25.257673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130ec2e-6bad-481f-8afa-e2e8721b3309 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:25.257673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130ec2e-6bad-481f-8afa-e2e8721b3309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:25.277883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7130ec2e-6bad-481f-8afa-e2e8721b3309 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:25.279737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7130ec2e-6bad-481f-8afa-e2e8721b3309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:25.362635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 571/1136] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:25.387136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc3653-8a1d-4543-9e2a-44e639d82920 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:25.391540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:25.505615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc3653-8a1d-4543-9e2a-44e639d82920 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:25.542091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8f909-eb17-487a-825b-736372d671a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:25.542735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8f909-eb17-487a-825b-736372d671a0 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:25.543017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8f909-eb17-487a-825b-736372d671a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:25.543286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8f909-eb17-487a-825b-736372d671a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:25.544740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8f909-eb17-487a-825b-736372d671a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:25.546193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8f909-eb17-487a-825b-736372d671a0 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:25.547879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 566/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:52:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:25.559013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130ec2e-6bad-481f-8afa-e2e8721b3309 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:25.673251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130ec2e-6bad-481f-8afa-e2e8721b3309 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:25.695199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:25.712924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7578cdb-d9a3-43b5-b3af-bd0adf979ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:25.713555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578cdb-d9a3-43b5-b3af-bd0adf979ff6 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:25.713834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578cdb-d9a3-43b5-b3af-bd0adf979ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:25.714106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7578cdb-d9a3-43b5-b3af-bd0adf979ff6 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:25.714837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7578cdb-d9a3-43b5-b3af-bd0adf979ff6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:25.716321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7578cdb-d9a3-43b5-b3af-bd0adf979ff6 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:25.744604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 572/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:52:25 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:52:27.790743 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438fa40-231d-4809-abac-f539331d28c7 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:27.791316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438fa40-231d-4809-abac-f539331d28c7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:27.791316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b438fa40-231d-4809-abac-f539331d28c7 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:27.792703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 567/1139] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:27.798377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed967af-7fef-4be9-99d0-b0bd968be8be None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:27.798868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed967af-7fef-4be9-99d0-b0bd968be8be None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:27.798868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed967af-7fef-4be9-99d0-b0bd968be8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:27.817361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2ed967af-7fef-4be9-99d0-b0bd968be8be None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:27.818539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2ed967af-7fef-4be9-99d0-b0bd968be8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:27.886920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 573/1140] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:52:27.910235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438fa40-231d-4809-abac-f539331d28c7 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:27.988308 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:28.020037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b438fa40-231d-4809-abac-f539331d28c7 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:28.058844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703bd9f-a437-4032-af07-3f9974a50e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:28.059464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703bd9f-a437-4032-af07-3f9974a50e29 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:28.059676 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703bd9f-a437-4032-af07-3f9974a50e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:28.059945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703bd9f-a437-4032-af07-3f9974a50e29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:28.072922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703bd9f-a437-4032-af07-3f9974a50e29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-0703bd9f-a437-4032-af07-3f9974a50e29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:28.078071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:28.079470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:28.080823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:28.084262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 568/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:28.085481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed967af-7fef-4be9-99d0-b0bd968be8be None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:28.189500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed967af-7fef-4be9-99d0-b0bd968be8be None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:28.197579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:28.228549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746d513-2b59-45b5-96af-0f2a54de0883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:28.229241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746d513-2b59-45b5-96af-0f2a54de0883 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:28.229521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746d513-2b59-45b5-96af-0f2a54de0883 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:28.229797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746d513-2b59-45b5-96af-0f2a54de0883 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:28.230498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746d513-2b59-45b5-96af-0f2a54de0883 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:28.230808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746d513-2b59-45b5-96af-0f2a54de0883 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:28.232583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746d513-2b59-45b5-96af-0f2a54de0883 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:28.240847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 574/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:28.246894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703bd9f-a437-4032-af07-3f9974a50e29 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:28.362261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703bd9f-a437-4032-af07-3f9974a50e29 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:28.392276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:28.398333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6428c-00e2-452f-bb4b-40d6602234f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:28.398985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6428c-00e2-452f-bb4b-40d6602234f8 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:28.399270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6428c-00e2-452f-bb4b-40d6602234f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:28.399548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a6428c-00e2-452f-bb4b-40d6602234f8 None admin] PATH_INFO: `/v3/users/a813094812ef400381aaf5ed25ab65ef` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:28.415265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a6428c-00e2-452f-bb4b-40d6602234f8 None admin] RBAC: Authorizing `identity:get_user(user_id=a813094812ef400381aaf5ed25ab65ef)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:28.416870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a6428c-00e2-452f-bb4b-40d6602234f8 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:28.418549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 569/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:28 2022] GET /identity/v3/users/a813094812ef400381aaf5ed25ab65ef => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:28.424543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746d513-2b59-45b5-96af-0f2a54de0883 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:28.532907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746d513-2b59-45b5-96af-0f2a54de0883 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:28.572367 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19f7e3-8ecc-4fda-8c36-9116937c9daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:28.572993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19f7e3-8ecc-4fda-8c36-9116937c9daf None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:28.573270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19f7e3-8ecc-4fda-8c36-9116937c9daf None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:28.573539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19f7e3-8ecc-4fda-8c36-9116937c9daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:28.621580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:28.659969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19f7e3-8ecc-4fda-8c36-9116937c9daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:28.661474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19f7e3-8ecc-4fda-8c36-9116937c9daf None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:28.774387 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 575/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:28.781530 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6428c-00e2-452f-bb4b-40d6602234f8 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:28.903892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a6428c-00e2-452f-bb4b-40d6602234f8 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:28.939323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79393049-b300-4918-a0c7-8026165a50b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:28.939964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79393049-b300-4918-a0c7-8026165a50b6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:28.940248 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79393049-b300-4918-a0c7-8026165a50b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:28.940555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79393049-b300-4918-a0c7-8026165a50b6 None admin] PATH_INFO: `/v3/projects/d80e93206b6640e99660338adb186ec4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:28.941733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79393049-b300-4918-a0c7-8026165a50b6 None admin] RBAC: Authorizing `identity:get_project(project_id=d80e93206b6640e99660338adb186ec4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:28.943204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79393049-b300-4918-a0c7-8026165a50b6 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:28.944946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 570/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:28 2022] GET /identity/v3/projects/d80e93206b6640e99660338adb186ec4 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:28.955044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19f7e3-8ecc-4fda-8c36-9116937c9daf None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:29.042704 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:29.086909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19f7e3-8ecc-4fda-8c36-9116937c9daf None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:29.124282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e705ea0b-35f5-438f-a904-c731243a2764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:29.124931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705ea0b-35f5-438f-a904-c731243a2764 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:29.125246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705ea0b-35f5-438f-a904-c731243a2764 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:29.125532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705ea0b-35f5-438f-a904-c731243a2764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:29.126326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705ea0b-35f5-438f-a904-c731243a2764 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=d80e93206b6640e99660338adb186ec4, user.id=a813094812ef400381aaf5ed25ab65ef) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:29.126636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705ea0b-35f5-438f-a904-c731243a2764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:29.128088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e705ea0b-35f5-438f-a904-c731243a2764 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:29.133837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 576/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:52:28 2022] GET /identity/v3/role_assignments?user.id=a813094812ef400381aaf5ed25ab65ef&scope.project.id=d80e93206b6640e99660338adb186ec4&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:31.079776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abebc9-f5d9-4789-b385-99d1d8c78196 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:31.080479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abebc9-f5d9-4789-b385-99d1d8c78196 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:31.080605 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abebc9-f5d9-4789-b385-99d1d8c78196 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:31.082587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 571/1147] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:31.091413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81db50-83fc-48b9-971c-e2beac145c7d None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:31.091941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81db50-83fc-48b9-971c-e2beac145c7d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:31.091941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81db50-83fc-48b9-971c-e2beac145c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:31.112633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-5b81db50-83fc-48b9-971c-e2beac145c7d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:31.113769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-5b81db50-83fc-48b9-971c-e2beac145c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:31.185513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 577/1148] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:31.208598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abebc9-f5d9-4789-b385-99d1d8c78196 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:31.275105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:31.322756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abebc9-f5d9-4789-b385-99d1d8c78196 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:31.360174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04a3d8-8a91-4af3-9f16-655821b531e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:31.360840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04a3d8-8a91-4af3-9f16-655821b531e3 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:31.361117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04a3d8-8a91-4af3-9f16-655821b531e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:31.361388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04a3d8-8a91-4af3-9f16-655821b531e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:31.372870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04a3d8-8a91-4af3-9f16-655821b531e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-cb04a3d8-8a91-4af3-9f16-655821b531e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:31.377876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:31.378945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:31.379972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 572/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:52:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:31.385584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81db50-83fc-48b9-971c-e2beac145c7d None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:31.458137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:31.510699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81db50-83fc-48b9-971c-e2beac145c7d None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:31.553196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca79995-4d3e-41c8-b0e4-da0038ff9ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:31.553815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca79995-4d3e-41c8-b0e4-da0038ff9ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:31.554087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca79995-4d3e-41c8-b0e4-da0038ff9ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:31.554355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca79995-4d3e-41c8-b0e4-da0038ff9ee5 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:31.555069 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca79995-4d3e-41c8-b0e4-da0038ff9ee5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:31.555364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca79995-4d3e-41c8-b0e4-da0038ff9ee5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:31.556785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca79995-4d3e-41c8-b0e4-da0038ff9ee5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:31.565031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 578/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:52:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:31.574861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04a3d8-8a91-4af3-9f16-655821b531e3 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:31.703643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04a3d8-8a91-4af3-9f16-655821b531e3 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:31.721286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:31.741517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25658ac4-f5fd-4c28-8557-caf6ad94b8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:31.742172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25658ac4-f5fd-4c28-8557-caf6ad94b8ad None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:31.742453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25658ac4-f5fd-4c28-8557-caf6ad94b8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:31.742739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25658ac4-f5fd-4c28-8557-caf6ad94b8ad None admin] PATH_INFO: `/v3/users/a813094812ef400381aaf5ed25ab65ef` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:31.743917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25658ac4-f5fd-4c28-8557-caf6ad94b8ad None admin] RBAC: Authorizing `identity:get_user(user_id=a813094812ef400381aaf5ed25ab65ef)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:31.745435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25658ac4-f5fd-4c28-8557-caf6ad94b8ad None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:31.747075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 573/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:31 2022] GET /identity/v3/users/a813094812ef400381aaf5ed25ab65ef => generated 286 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:31.753360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca79995-4d3e-41c8-b0e4-da0038ff9ee5 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:31.862685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca79995-4d3e-41c8-b0e4-da0038ff9ee5 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:31.886537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:31.900374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889075e7-b398-4474-8cb9-c6095ee9f3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:31.901025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889075e7-b398-4474-8cb9-c6095ee9f3be None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:31.901309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889075e7-b398-4474-8cb9-c6095ee9f3be None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:31.901580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889075e7-b398-4474-8cb9-c6095ee9f3be None admin] PATH_INFO: `/v3/projects/d80e93206b6640e99660338adb186ec4` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:31.902750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889075e7-b398-4474-8cb9-c6095ee9f3be None admin] RBAC: Authorizing `identity:get_project(project_id=d80e93206b6640e99660338adb186ec4)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:31.904202 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889075e7-b398-4474-8cb9-c6095ee9f3be None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:31.905947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 579/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:31 2022] GET /identity/v3/projects/d80e93206b6640e99660338adb186ec4 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:31.915219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25658ac4-f5fd-4c28-8557-caf6ad94b8ad None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:32.035395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25658ac4-f5fd-4c28-8557-caf6ad94b8ad None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:32.071952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c45747-69ca-44ab-84b0-99e303a67a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:32.072645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45747-69ca-44ab-84b0-99e303a67a4c None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:32.072923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45747-69ca-44ab-84b0-99e303a67a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:32.073194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c45747-69ca-44ab-84b0-99e303a67a4c None admin] PATH_INFO: `/v3/projects/d80e93206b6640e99660338adb186ec4/users/a813094812ef400381aaf5ed25ab65ef/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:32.075145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c45747-69ca-44ab-84b0-99e303a67a4c None admin] RBAC: Authorizing `identity:create_grant(project_id=d80e93206b6640e99660338adb186ec4, user_id=a813094812ef400381aaf5ed25ab65ef, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:32.076619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c45747-69ca-44ab-84b0-99e303a67a4c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:32.095294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 574/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:52:31 2022] PUT /identity/v3/projects/d80e93206b6640e99660338adb186ec4/users/a813094812ef400381aaf5ed25ab65ef/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:52:34.042088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc46cf5-f1cf-4683-b028-e685ae322510 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:34.042692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc46cf5-f1cf-4683-b028-e685ae322510 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:34.042692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc46cf5-f1cf-4683-b028-e685ae322510 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:34.043977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 580/1154] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:34.050525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267feda5-c374-4fed-a62c-96d7e2ee883d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:34.051229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267feda5-c374-4fed-a62c-96d7e2ee883d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:34.051419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267feda5-c374-4fed-a62c-96d7e2ee883d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:34.080870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-267feda5-c374-4fed-a62c-96d7e2ee883d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:34.083830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-267feda5-c374-4fed-a62c-96d7e2ee883d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:34.132636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:34.159614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 575/1155] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:34.183417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc46cf5-f1cf-4683-b028-e685ae322510 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:34.308959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc46cf5-f1cf-4683-b028-e685ae322510 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:34.344509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec58b4-7329-4afc-9da0-4d78656a010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:34.345154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec58b4-7329-4afc-9da0-4d78656a010c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:34.345432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec58b4-7329-4afc-9da0-4d78656a010c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:34.345710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec58b4-7329-4afc-9da0-4d78656a010c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:34.357886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec58b4-7329-4afc-9da0-4d78656a010c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-91ec58b4-7329-4afc-9da0-4d78656a010c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:34.362834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:34.363884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:52:34.365136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 581/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:52:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:34.373441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267feda5-c374-4fed-a62c-96d7e2ee883d None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:34.479941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267feda5-c374-4fed-a62c-96d7e2ee883d None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:34.517543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ed0a-70d6-42be-9b75-c0fea5cf6286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:34.518172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ed0a-70d6-42be-9b75-c0fea5cf6286 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:34.518453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ed0a-70d6-42be-9b75-c0fea5cf6286 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:34.518721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4ed0a-70d6-42be-9b75-c0fea5cf6286 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:34.519454 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4ed0a-70d6-42be-9b75-c0fea5cf6286 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:34.519755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4ed0a-70d6-42be-9b75-c0fea5cf6286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:34.521174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4ed0a-70d6-42be-9b75-c0fea5cf6286 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:34.530670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 576/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:52:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:34.540779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec58b4-7329-4afc-9da0-4d78656a010c None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:34.600641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:34.652966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec58b4-7329-4afc-9da0-4d78656a010c None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:34.690241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be17ea8-44c7-46d5-aabf-9319068d93f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:34.690888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be17ea8-44c7-46d5-aabf-9319068d93f1 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:34.691149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be17ea8-44c7-46d5-aabf-9319068d93f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:34.691421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be17ea8-44c7-46d5-aabf-9319068d93f1 None admin] PATH_INFO: `/v3/users/a813094812ef400381aaf5ed25ab65ef` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:34.692639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be17ea8-44c7-46d5-aabf-9319068d93f1 None admin] RBAC: Authorizing `identity:get_user(user_id=a813094812ef400381aaf5ed25ab65ef)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:34.694082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be17ea8-44c7-46d5-aabf-9319068d93f1 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:34.695706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 582/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:34 2022] GET /identity/v3/users/a813094812ef400381aaf5ed25ab65ef => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:34.704182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ed0a-70d6-42be-9b75-c0fea5cf6286 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:34.746670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:34.810397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4ed0a-70d6-42be-9b75-c0fea5cf6286 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:34.845700 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888c17f-6073-4f3e-a972-920323003823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:34.846321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888c17f-6073-4f3e-a972-920323003823 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:34.846601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888c17f-6073-4f3e-a972-920323003823 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:34.846870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888c17f-6073-4f3e-a972-920323003823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:34.922446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888c17f-6073-4f3e-a972-920323003823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:34.923911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888c17f-6073-4f3e-a972-920323003823 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:34.963939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:35.029129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 577/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:35.035480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be17ea8-44c7-46d5-aabf-9319068d93f1 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:35.149439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be17ea8-44c7-46d5-aabf-9319068d93f1 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:35.171019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:35.187692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779c48e-b347-46ad-a014-e02d816e3d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:35.188333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779c48e-b347-46ad-a014-e02d816e3d5d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:35.188650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779c48e-b347-46ad-a014-e02d816e3d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:35.188925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779c48e-b347-46ad-a014-e02d816e3d5d None admin] PATH_INFO: `/v3/projects/d80e93206b6640e99660338adb186ec4` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:35.190103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779c48e-b347-46ad-a014-e02d816e3d5d None admin] RBAC: Authorizing `identity:get_project(project_id=d80e93206b6640e99660338adb186ec4)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:35.191557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779c48e-b347-46ad-a014-e02d816e3d5d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:35.193276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 583/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:35 2022] GET /identity/v3/projects/d80e93206b6640e99660338adb186ec4 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:35.199860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888c17f-6073-4f3e-a972-920323003823 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:35.305638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888c17f-6073-4f3e-a972-920323003823 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:35.341186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe6ee9-8c11-490d-bd25-6689a18e180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:35.341810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe6ee9-8c11-490d-bd25-6689a18e180e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:35.342092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe6ee9-8c11-490d-bd25-6689a18e180e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:35.342366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe6ee9-8c11-490d-bd25-6689a18e180e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:35.343158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe6ee9-8c11-490d-bd25-6689a18e180e None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=d80e93206b6640e99660338adb186ec4, user.id=a813094812ef400381aaf5ed25ab65ef) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:35.343594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe6ee9-8c11-490d-bd25-6689a18e180e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:35.345082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe6ee9-8c11-490d-bd25-6689a18e180e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:35.351276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 578/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:52:35 2022] GET /identity/v3/role_assignments?user.id=a813094812ef400381aaf5ed25ab65ef&scope.project.id=d80e93206b6640e99660338adb186ec4&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:37.324359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c688f2-18b8-4ccf-9b30-32d7bd490c6c None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:37.325214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c688f2-18b8-4ccf-9b30-32d7bd490c6c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:37.325214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c688f2-18b8-4ccf-9b30-32d7bd490c6c None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:37.327317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 584/1162] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:37.333594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c6fed-0597-4a57-a8b7-c7491358ff47 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:37.334264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c6fed-0597-4a57-a8b7-c7491358ff47 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:37.334386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c6fed-0597-4a57-a8b7-c7491358ff47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:37.344600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:37.358739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d12c6fed-0597-4a57-a8b7-c7491358ff47 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:37.359972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d12c6fed-0597-4a57-a8b7-c7491358ff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:37.434906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 579/1163] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:37.457744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c688f2-18b8-4ccf-9b30-32d7bd490c6c None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:37.576115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c688f2-18b8-4ccf-9b30-32d7bd490c6c None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:37.611466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e365b6-6e54-41e8-af89-a5a59dfcc993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:37.612082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e365b6-6e54-41e8-af89-a5a59dfcc993 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:37.612356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e365b6-6e54-41e8-af89-a5a59dfcc993 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:37.612653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e365b6-6e54-41e8-af89-a5a59dfcc993 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:37.675350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:37.695179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e365b6-6e54-41e8-af89-a5a59dfcc993 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:37.697510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e365b6-6e54-41e8-af89-a5a59dfcc993 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:37.812045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 585/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:37.818424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c6fed-0597-4a57-a8b7-c7491358ff47 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:37.935065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c6fed-0597-4a57-a8b7-c7491358ff47 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:37.968098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:37.975458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bebef-5347-4d25-a1e6-6d980af0ca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:37.976118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bebef-5347-4d25-a1e6-6d980af0ca92 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:37.976437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bebef-5347-4d25-a1e6-6d980af0ca92 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:37.976716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bebef-5347-4d25-a1e6-6d980af0ca92 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:37.984580 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bebef-5347-4d25-a1e6-6d980af0ca92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:37.986044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bebef-5347-4d25-a1e6-6d980af0ca92 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-ba9bebef-5347-4d25-a1e6-6d980af0ca92 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:37.999208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:38.000448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 01 11:52:38.001518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:38.004444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 580/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Aug 1 11:52:37 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:38.007087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e365b6-6e54-41e8-af89-a5a59dfcc993 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:38.114917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:38.131209 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e365b6-6e54-41e8-af89-a5a59dfcc993 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:38.168243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d03167-f716-4b4c-aacb-d82562569cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:38.168888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d03167-f716-4b4c-aacb-d82562569cfd None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:38.169165 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d03167-f716-4b4c-aacb-d82562569cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:38.169432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d03167-f716-4b4c-aacb-d82562569cfd None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:38.170126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d03167-f716-4b4c-aacb-d82562569cfd None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:38.170424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d03167-f716-4b4c-aacb-d82562569cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:38.171794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d03167-f716-4b4c-aacb-d82562569cfd None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:38.179045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 586/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Aug 1 11:52:38 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:40.159810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a1141-e112-4706-b3b5-176d2b18a2c3 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:40.160397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a1141-e112-4706-b3b5-176d2b18a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:40.160397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965a1141-e112-4706-b3b5-176d2b18a2c3 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:40.161656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 581/1167] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:40.167510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59924f-5b1b-490f-8516-283dd7971668 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:40.168233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59924f-5b1b-490f-8516-283dd7971668 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:40.168443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59924f-5b1b-490f-8516-283dd7971668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:40.194760 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-1e59924f-5b1b-490f-8516-283dd7971668 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:40.195975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1e59924f-5b1b-490f-8516-283dd7971668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:40.270213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 587/1168] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:40.293125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a1141-e112-4706-b3b5-176d2b18a2c3 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:40.404460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965a1141-e112-4706-b3b5-176d2b18a2c3 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:40.442575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd45000d-c33a-4187-a6ad-de64f3cf2679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:40.443141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45000d-c33a-4187-a6ad-de64f3cf2679 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:40.443417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45000d-c33a-4187-a6ad-de64f3cf2679 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:40.443692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd45000d-c33a-4187-a6ad-de64f3cf2679 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:40.444444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45000d-c33a-4187-a6ad-de64f3cf2679 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:40.445834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd45000d-c33a-4187-a6ad-de64f3cf2679 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:40.469232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 582/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Mon Aug 1 11:52:40 2022] POST /identity/v3/domains => generated 267 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:52:42.494318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385136d2-eccc-4044-a3f2-060fdf86057e None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:42.495220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385136d2-eccc-4044-a3f2-060fdf86057e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:42.495220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385136d2-eccc-4044-a3f2-060fdf86057e None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:42.497426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 588/1170] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:42.503705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915de519-041a-4b06-aaba-fb76b6ec0f74 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:42.504155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915de519-041a-4b06-aaba-fb76b6ec0f74 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:42.504265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915de519-041a-4b06-aaba-fb76b6ec0f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:42.523830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-915de519-041a-4b06-aaba-fb76b6ec0f74 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:42.525083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-915de519-041a-4b06-aaba-fb76b6ec0f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:42.579141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:42.603382 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 583/1171] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:52:42.634575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385136d2-eccc-4044-a3f2-060fdf86057e None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:42.726969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:42.748837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385136d2-eccc-4044-a3f2-060fdf86057e None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:42.786558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56947b33-9332-4830-b84e-700473bd80e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:42.787198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56947b33-9332-4830-b84e-700473bd80e5 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:42.787475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56947b33-9332-4830-b84e-700473bd80e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:42.787772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56947b33-9332-4830-b84e-700473bd80e5 None admin] PATH_INFO: `/v3/domains/25d7966965644c90a9e6b78d41e1b7ce` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:42.798154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56947b33-9332-4830-b84e-700473bd80e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=25d7966965644c90a9e6b78d41e1b7ce)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:42.799604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56947b33-9332-4830-b84e-700473bd80e5 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:42.801262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 589/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Mon Aug 1 11:52:42 2022] GET /identity/v3/domains/25d7966965644c90a9e6b78d41e1b7ce => generated 267 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:42.808093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915de519-041a-4b06-aaba-fb76b6ec0f74 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:42.925182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915de519-041a-4b06-aaba-fb76b6ec0f74 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:42.962488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815db67-dfdb-491f-8525-d356f125f4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:42.963105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815db67-dfdb-491f-8525-d356f125f4f2 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:42.963381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815db67-dfdb-491f-8525-d356f125f4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:42.963649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815db67-dfdb-491f-8525-d356f125f4f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:42.964350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815db67-dfdb-491f-8525-d356f125f4f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:42.965820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815db67-dfdb-491f-8525-d356f125f4f2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:42.998932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 584/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:52:42 2022] POST /identity/v3/projects => generated 361 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:52:45.027682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94b9bc-1f9e-4514-8904-cb6eaa477281 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:45.028241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94b9bc-1f9e-4514-8904-cb6eaa477281 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:45.028241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94b9bc-1f9e-4514-8904-cb6eaa477281 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:45.029651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 590/1174] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:45.036023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df376254-7f7e-4426-980f-0be5301ae332 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:45.036936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df376254-7f7e-4426-980f-0be5301ae332 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:45.036936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df376254-7f7e-4426-980f-0be5301ae332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:45.060263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-df376254-7f7e-4426-980f-0be5301ae332 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:45.061494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-df376254-7f7e-4426-980f-0be5301ae332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:45.090385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:45.134878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 585/1175] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:45.159039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94b9bc-1f9e-4514-8904-cb6eaa477281 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:45.274983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94b9bc-1f9e-4514-8904-cb6eaa477281 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:45.319958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0de66-a316-42aa-9de4-266b50dcb197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:45.320927 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0de66-a316-42aa-9de4-266b50dcb197 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:45.321302 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0de66-a316-42aa-9de4-266b50dcb197 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:45.321682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0de66-a316-42aa-9de4-266b50dcb197 None admin] PATH_INFO: `/v3/domains/25d7966965644c90a9e6b78d41e1b7ce` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:45.323707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0de66-a316-42aa-9de4-266b50dcb197 None admin] RBAC: Authorizing `identity:get_domain(domain_id=25d7966965644c90a9e6b78d41e1b7ce)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:45.326006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0de66-a316-42aa-9de4-266b50dcb197 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:45.328116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 591/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Mon Aug 1 11:52:45 2022] GET /identity/v3/domains/25d7966965644c90a9e6b78d41e1b7ce => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:45.336653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df376254-7f7e-4426-980f-0be5301ae332 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:45.441169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df376254-7f7e-4426-980f-0be5301ae332 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:45.476813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e685c-c9cd-4277-9f1a-5847d7728b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:45.477468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e685c-c9cd-4277-9f1a-5847d7728b51 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:45.477722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e685c-c9cd-4277-9f1a-5847d7728b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:45.478022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e685c-c9cd-4277-9f1a-5847d7728b51 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:45.478710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e685c-c9cd-4277-9f1a-5847d7728b51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:45.480160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e685c-c9cd-4277-9f1a-5847d7728b51 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:45.504810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 586/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:52:45 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:52:47.540690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f19ffb-7b73-4ea7-adfe-767a38b39003 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:47.541309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f19ffb-7b73-4ea7-adfe-767a38b39003 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:47.541309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f19ffb-7b73-4ea7-adfe-767a38b39003 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:47.543266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 592/1178] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:47.549696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd620bdb-731f-49ac-910c-ff32093ca935 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:47.550208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd620bdb-731f-49ac-910c-ff32093ca935 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:47.550208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd620bdb-731f-49ac-910c-ff32093ca935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:47.568698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-cd620bdb-731f-49ac-910c-ff32093ca935 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:47.569884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-cd620bdb-731f-49ac-910c-ff32093ca935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:47.583986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:47.640197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 587/1179] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:52:47.663261 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f19ffb-7b73-4ea7-adfe-767a38b39003 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:47.678025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:47.793439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f19ffb-7b73-4ea7-adfe-767a38b39003 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:47.830650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8808254-287b-4c58-b6eb-c260ab6bc2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:47.831307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8808254-287b-4c58-b6eb-c260ab6bc2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:47.831590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8808254-287b-4c58-b6eb-c260ab6bc2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:47.831861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8808254-287b-4c58-b6eb-c260ab6bc2a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:47.843944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8808254-287b-4c58-b6eb-c260ab6bc2a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-d8808254-287b-4c58-b6eb-c260ab6bc2a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:47.849455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:47.850539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:52:47.851674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 593/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:52:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:47.857144 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd620bdb-731f-49ac-910c-ff32093ca935 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:47.969181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd620bdb-731f-49ac-910c-ff32093ca935 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:48.007571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b2f61-2536-4d80-b80e-375816088ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:48.008200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b2f61-2536-4d80-b80e-375816088ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:48.008512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b2f61-2536-4d80-b80e-375816088ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:48.008784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b2f61-2536-4d80-b80e-375816088ab4 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:48.009505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b2f61-2536-4d80-b80e-375816088ab4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:48.009810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b2f61-2536-4d80-b80e-375816088ab4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:48.011203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b2f61-2536-4d80-b80e-375816088ab4 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:48.019627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 588/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:52:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:48.025692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8808254-287b-4c58-b6eb-c260ab6bc2a9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:48.102025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:48.135247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8808254-287b-4c58-b6eb-c260ab6bc2a9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:48.170525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df245d-725c-4731-86fa-401aa4140cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:48.171229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df245d-725c-4731-86fa-401aa4140cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:48.171510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df245d-725c-4731-86fa-401aa4140cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:48.171784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df245d-725c-4731-86fa-401aa4140cd8 None admin] PATH_INFO: `/v3/users/6dfee78d1ecc4c67840971b892141729` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:48.185393 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df245d-725c-4731-86fa-401aa4140cd8 None admin] RBAC: Authorizing `identity:get_user(user_id=6dfee78d1ecc4c67840971b892141729)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:48.186885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df245d-725c-4731-86fa-401aa4140cd8 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:48.188552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 594/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:48 2022] GET /identity/v3/users/6dfee78d1ecc4c67840971b892141729 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:48.195303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b2f61-2536-4d80-b80e-375816088ab4 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:48.215434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:48.320399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b2f61-2536-4d80-b80e-375816088ab4 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:48.355586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0f783-f4da-4dc8-b1d9-c2b0b690e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:48.356132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0f783-f4da-4dc8-b1d9-c2b0b690e3fa None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:48.356446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0f783-f4da-4dc8-b1d9-c2b0b690e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:48.356717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b0f783-f4da-4dc8-b1d9-c2b0b690e3fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:48.432976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0f783-f4da-4dc8-b1d9-c2b0b690e3fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:48.434461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b0f783-f4da-4dc8-b1d9-c2b0b690e3fa None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:48.448324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:48.549494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 589/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:48.556122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df245d-725c-4731-86fa-401aa4140cd8 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:48.668282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df245d-725c-4731-86fa-401aa4140cd8 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:48.695965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:48.705187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf945a2f-e186-4c50-a95e-a589bf6f0ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:48.705838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf945a2f-e186-4c50-a95e-a589bf6f0ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:48.706115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf945a2f-e186-4c50-a95e-a589bf6f0ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:48.706386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf945a2f-e186-4c50-a95e-a589bf6f0ac0 None admin] PATH_INFO: `/v3/projects/e587658da24540a7be2625e6d8a9a5b6` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:48.707569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf945a2f-e186-4c50-a95e-a589bf6f0ac0 None admin] RBAC: Authorizing `identity:get_project(project_id=e587658da24540a7be2625e6d8a9a5b6)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:48.709066 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf945a2f-e186-4c50-a95e-a589bf6f0ac0 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:48.710749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 595/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:48 2022] GET /identity/v3/projects/e587658da24540a7be2625e6d8a9a5b6 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:48.717252 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0f783-f4da-4dc8-b1d9-c2b0b690e3fa None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:48.826888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b0f783-f4da-4dc8-b1d9-c2b0b690e3fa None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:48.842088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:48.868729 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f99360-daf0-4734-ad64-05e48d41a717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:48.869357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f99360-daf0-4734-ad64-05e48d41a717 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:48.869637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f99360-daf0-4734-ad64-05e48d41a717 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:48.869908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f99360-daf0-4734-ad64-05e48d41a717 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:48.870707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f99360-daf0-4734-ad64-05e48d41a717 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=e587658da24540a7be2625e6d8a9a5b6, user.id=6dfee78d1ecc4c67840971b892141729) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:48.871016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f99360-daf0-4734-ad64-05e48d41a717 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:48.872486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f99360-daf0-4734-ad64-05e48d41a717 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:48.878023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 590/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:52:48 2022] GET /identity/v3/role_assignments?user.id=6dfee78d1ecc4c67840971b892141729&scope.project.id=e587658da24540a7be2625e6d8a9a5b6&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:50.859114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785165f-0978-4be3-8848-ae51b4e9ae65 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:50.859619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785165f-0978-4be3-8848-ae51b4e9ae65 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:50.859619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785165f-0978-4be3-8848-ae51b4e9ae65 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:50.861018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 596/1186] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:50.866579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607f060-e459-4cf4-854a-cc611a814993 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:50.867069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607f060-e459-4cf4-854a-cc611a814993 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:50.867069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6607f060-e459-4cf4-854a-cc611a814993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:50.887028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-6607f060-e459-4cf4-854a-cc611a814993 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:50.888258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6607f060-e459-4cf4-854a-cc611a814993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:50.960594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 591/1187] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:50.983432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785165f-0978-4be3-8848-ae51b4e9ae65 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:51.095410 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785165f-0978-4be3-8848-ae51b4e9ae65 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:51.131457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b510741-cc7c-4cfe-97a2-43e978458853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:51.132007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b510741-cc7c-4cfe-97a2-43e978458853 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:51.132320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b510741-cc7c-4cfe-97a2-43e978458853 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:51.132603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b510741-cc7c-4cfe-97a2-43e978458853 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:51.143047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b510741-cc7c-4cfe-97a2-43e978458853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-0b510741-cc7c-4cfe-97a2-43e978458853 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:51.148000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:51.149144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:52:51.150071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 597/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:52:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:51.155430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607f060-e459-4cf4-854a-cc611a814993 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:51.260572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6607f060-e459-4cf4-854a-cc611a814993 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:51.268445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:51.297298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7cf5-08f4-42ed-9b56-0556249b8309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:51.297925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7cf5-08f4-42ed-9b56-0556249b8309 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:51.298203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7cf5-08f4-42ed-9b56-0556249b8309 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:51.298476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7cf5-08f4-42ed-9b56-0556249b8309 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:51.299200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd7cf5-08f4-42ed-9b56-0556249b8309 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:51.299503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd7cf5-08f4-42ed-9b56-0556249b8309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:51.300929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd7cf5-08f4-42ed-9b56-0556249b8309 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:51.309227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 592/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:52:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:51.315304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b510741-cc7c-4cfe-97a2-43e978458853 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:51.372972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:51.419967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b510741-cc7c-4cfe-97a2-43e978458853 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:51.454849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5a9ff-16db-43a9-a34c-5abd3953a00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:51.455495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a9ff-16db-43a9-a34c-5abd3953a00b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:51.455780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a9ff-16db-43a9-a34c-5abd3953a00b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:51.456041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f5a9ff-16db-43a9-a34c-5abd3953a00b None admin] PATH_INFO: `/v3/users/6dfee78d1ecc4c67840971b892141729` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:51.457241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5a9ff-16db-43a9-a34c-5abd3953a00b None admin] RBAC: Authorizing `identity:get_user(user_id=6dfee78d1ecc4c67840971b892141729)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:51.458700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f5a9ff-16db-43a9-a34c-5abd3953a00b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:51.460339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 598/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:51 2022] GET /identity/v3/users/6dfee78d1ecc4c67840971b892141729 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:51.466519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7cf5-08f4-42ed-9b56-0556249b8309 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:51.572118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7cf5-08f4-42ed-9b56-0556249b8309 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:51.606785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e13332c-c19d-49af-a597-ffbabe0c75e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:51.607435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13332c-c19d-49af-a597-ffbabe0c75e7 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:51.607727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13332c-c19d-49af-a597-ffbabe0c75e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:51.608001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13332c-c19d-49af-a597-ffbabe0c75e7 None admin] PATH_INFO: `/v3/projects/e587658da24540a7be2625e6d8a9a5b6` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:51.609294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13332c-c19d-49af-a597-ffbabe0c75e7 None admin] RBAC: Authorizing `identity:get_project(project_id=e587658da24540a7be2625e6d8a9a5b6)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:51.610769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e13332c-c19d-49af-a597-ffbabe0c75e7 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:51.612447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 593/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:51 2022] GET /identity/v3/projects/e587658da24540a7be2625e6d8a9a5b6 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:51.622097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5a9ff-16db-43a9-a34c-5abd3953a00b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:51.726692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f5a9ff-16db-43a9-a34c-5abd3953a00b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:51.751989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:51.763370 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafcdcd-18bb-402f-880e-cf6cf3b07c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:51.764128 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafcdcd-18bb-402f-880e-cf6cf3b07c03 None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:51.764467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafcdcd-18bb-402f-880e-cf6cf3b07c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:51.764741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafcdcd-18bb-402f-880e-cf6cf3b07c03 None admin] PATH_INFO: `/v3/projects/e587658da24540a7be2625e6d8a9a5b6/users/6dfee78d1ecc4c67840971b892141729/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:51.766727 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafcdcd-18bb-402f-880e-cf6cf3b07c03 None admin] RBAC: Authorizing `identity:create_grant(project_id=e587658da24540a7be2625e6d8a9a5b6, user_id=6dfee78d1ecc4c67840971b892141729, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:51.768171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafcdcd-18bb-402f-880e-cf6cf3b07c03 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:51.782021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 599/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:52:51 2022] PUT /identity/v3/projects/e587658da24540a7be2625e6d8a9a5b6/users/6dfee78d1ecc4c67840971b892141729/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 11:52:53.722343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0f0c6-3224-4788-b52c-4b46cb95b2bf None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:53.722940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0f0c6-3224-4788-b52c-4b46cb95b2bf None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:53.722940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0f0c6-3224-4788-b52c-4b46cb95b2bf None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:53.724250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 594/1193] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:52:53.729714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cad6a-f6ed-409d-abc6-641186a85f50 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:53.730549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cad6a-f6ed-409d-abc6-641186a85f50 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:53.730649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cad6a-f6ed-409d-abc6-641186a85f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:53.750095 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8b3cad6a-f6ed-409d-abc6-641186a85f50 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:52:53.751313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8b3cad6a-f6ed-409d-abc6-641186a85f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:52:53.822708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 600/1194] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:52:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:52:53.846336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0f0c6-3224-4788-b52c-4b46cb95b2bf None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:53.884981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:53.954081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0f0c6-3224-4788-b52c-4b46cb95b2bf None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:53.990626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0b0cd-4064-485d-8f68-4f3bc3d4b505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:53.991175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0b0cd-4064-485d-8f68-4f3bc3d4b505 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:53.991455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0b0cd-4064-485d-8f68-4f3bc3d4b505 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:53.991727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0b0cd-4064-485d-8f68-4f3bc3d4b505 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:54.002390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0b0cd-4064-485d-8f68-4f3bc3d4b505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-99b0b0cd-4064-485d-8f68-4f3bc3d4b505 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:52:54.007337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:52:54.008440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:52:54.009436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:52:54.012483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 595/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:52:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:52:54.014794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cad6a-f6ed-409d-abc6-641186a85f50 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:54.118323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cad6a-f6ed-409d-abc6-641186a85f50 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:54.135173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:54.159576 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d87b02-6386-4051-bc17-8f0c25a5ef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:54.160199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d87b02-6386-4051-bc17-8f0c25a5ef22 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:54.160536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d87b02-6386-4051-bc17-8f0c25a5ef22 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:54.160791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d87b02-6386-4051-bc17-8f0c25a5ef22 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:54.161568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d87b02-6386-4051-bc17-8f0c25a5ef22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:54.161873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d87b02-6386-4051-bc17-8f0c25a5ef22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:54.163289 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d87b02-6386-4051-bc17-8f0c25a5ef22 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:54.173113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 601/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:52:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:52:54.179376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0b0cd-4064-485d-8f68-4f3bc3d4b505 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:54.290113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:54.297076 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0b0cd-4064-485d-8f68-4f3bc3d4b505 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:54.332342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8acd9-f6d9-49cd-9789-bc5ba19116f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:54.333007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8acd9-f6d9-49cd-9789-bc5ba19116f9 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:54.333281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8acd9-f6d9-49cd-9789-bc5ba19116f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:54.333548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8acd9-f6d9-49cd-9789-bc5ba19116f9 None admin] PATH_INFO: `/v3/users/6dfee78d1ecc4c67840971b892141729` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:54.334718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8acd9-f6d9-49cd-9789-bc5ba19116f9 None admin] RBAC: Authorizing `identity:get_user(user_id=6dfee78d1ecc4c67840971b892141729)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:54.336147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8acd9-f6d9-49cd-9789-bc5ba19116f9 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:54.337809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 596/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Aug 1 11:52:54 2022] GET /identity/v3/users/6dfee78d1ecc4c67840971b892141729 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:54.344044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d87b02-6386-4051-bc17-8f0c25a5ef22 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:54.449465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d87b02-6386-4051-bc17-8f0c25a5ef22 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:54.486591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09bfdb-c88b-4c46-991a-92cae4fdb4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:54.487220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09bfdb-c88b-4c46-991a-92cae4fdb4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:54.487498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09bfdb-c88b-4c46-991a-92cae4fdb4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:54.487771 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09bfdb-c88b-4c46-991a-92cae4fdb4e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:54.534787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:54.559250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09bfdb-c88b-4c46-991a-92cae4fdb4e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:54.560738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09bfdb-c88b-4c46-991a-92cae4fdb4e2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:54.666637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 602/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:52:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:52:54.672801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8acd9-f6d9-49cd-9789-bc5ba19116f9 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:54.786387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8acd9-f6d9-49cd-9789-bc5ba19116f9 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:54.824873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae813c-ea75-40c7-9ca0-25b694b4b94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:54.825439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae813c-ea75-40c7-9ca0-25b694b4b94c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:54.825719 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae813c-ea75-40c7-9ca0-25b694b4b94c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:54.826001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae813c-ea75-40c7-9ca0-25b694b4b94c None admin] PATH_INFO: `/v3/projects/e587658da24540a7be2625e6d8a9a5b6` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:54.827330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae813c-ea75-40c7-9ca0-25b694b4b94c None admin] RBAC: Authorizing `identity:get_project(project_id=e587658da24540a7be2625e6d8a9a5b6)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:54.828900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae813c-ea75-40c7-9ca0-25b694b4b94c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:54.830775 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 597/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Aug 1 11:52:54 2022] GET /identity/v3/projects/e587658da24540a7be2625e6d8a9a5b6 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:52:54.837356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09bfdb-c88b-4c46-991a-92cae4fdb4e2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:52:54.925894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:52:54.951866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09bfdb-c88b-4c46-991a-92cae4fdb4e2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:52:54.990079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68891e33-dd60-4b90-852c-024eb50ca909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:52:54.990712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68891e33-dd60-4b90-852c-024eb50ca909 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:52:54.990991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68891e33-dd60-4b90-852c-024eb50ca909 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:52:54.991262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68891e33-dd60-4b90-852c-024eb50ca909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:52:54.992075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68891e33-dd60-4b90-852c-024eb50ca909 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=e587658da24540a7be2625e6d8a9a5b6, user.id=6dfee78d1ecc4c67840971b892141729) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:52:54.992395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68891e33-dd60-4b90-852c-024eb50ca909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:52:54.993888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68891e33-dd60-4b90-852c-024eb50ca909 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:52:55.000904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 603/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:52:54 2022] GET /identity/v3/role_assignments?user.id=6dfee78d1ecc4c67840971b892141729&scope.project.id=e587658da24540a7be2625e6d8a9a5b6&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:53:04.820044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=84 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:04.997766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=30 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:52.355464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730e5e4-3294-4100-84f1-377aba69c730 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:52.356236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730e5e4-3294-4100-84f1-377aba69c730 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:52.356441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730e5e4-3294-4100-84f1-377aba69c730 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:52.358615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 598/1201] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:53:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:53:52.367487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0748e-3a3f-462c-af97-02db9fa6a48f None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:52.368142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0748e-3a3f-462c-af97-02db9fa6a48f None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:52.368142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0748e-3a3f-462c-af97-02db9fa6a48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:52.396323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c3c0748e-3a3f-462c-af97-02db9fa6a48f None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:53:52.397585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c3c0748e-3a3f-462c-af97-02db9fa6a48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:53:52.532349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 604/1202] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:53:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:53:52.567335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730e5e4-3294-4100-84f1-377aba69c730 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:52.758280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730e5e4-3294-4100-84f1-377aba69c730 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:52.825779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e3926-8657-4a9a-bedc-ba4b6290ed37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:52.826430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e3926-8657-4a9a-bedc-ba4b6290ed37 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:52.832972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e3926-8657-4a9a-bedc-ba4b6290ed37 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:52.833307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e3926-8657-4a9a-bedc-ba4b6290ed37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:52.856909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e3926-8657-4a9a-bedc-ba4b6290ed37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:52.859397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e3926-8657-4a9a-bedc-ba4b6290ed37 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-179e3926-8657-4a9a-bedc-ba4b6290ed37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:53:52.888843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:53:52.890520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:53:52.891977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:53:52.891977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:53:52.891977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:53:52.891977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:53:52.894911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 599/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:53:52.904855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0748e-3a3f-462c-af97-02db9fa6a48f None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:53.130642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c0748e-3a3f-462c-af97-02db9fa6a48f None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:53.198951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:53.209136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b34f0-ed2f-4ef7-b049-7af191b188b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:53.210062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b34f0-ed2f-4ef7-b049-7af191b188b9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:53.210775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b34f0-ed2f-4ef7-b049-7af191b188b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:53.211175 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b34f0-ed2f-4ef7-b049-7af191b188b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:53.212084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b34f0-ed2f-4ef7-b049-7af191b188b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:53:53.212548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b34f0-ed2f-4ef7-b049-7af191b188b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:53.214477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b34f0-ed2f-4ef7-b049-7af191b188b9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:53.233424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 605/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:53:53.243564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e3926-8657-4a9a-bedc-ba4b6290ed37 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:53.362625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:53.458923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e3926-8657-4a9a-bedc-ba4b6290ed37 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:53.533418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cbd79-0a8b-4fbc-8356-bd87616fe84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:53.535421 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cbd79-0a8b-4fbc-8356-bd87616fe84e None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:53.535909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cbd79-0a8b-4fbc-8356-bd87616fe84e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:53.536496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cbd79-0a8b-4fbc-8356-bd87616fe84e None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:53.537914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cbd79-0a8b-4fbc-8356-bd87616fe84e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:53.541140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cbd79-0a8b-4fbc-8356-bd87616fe84e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:53.591791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 600/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Aug 1 11:53:53 2022] POST /identity/v3/users => generated 251 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:53:56.488177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49ec0b-a9ee-44a8-b275-804263cc9dce None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:56.489030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49ec0b-a9ee-44a8-b275-804263cc9dce None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:56.489030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49ec0b-a9ee-44a8-b275-804263cc9dce None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:56.490546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 606/1206] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:53:56.496413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ac5d-654c-448a-be48-f8a394158d8e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:56.497240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ac5d-654c-448a-be48-f8a394158d8e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:56.497240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853ac5d-654c-448a-be48-f8a394158d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:56.519581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-7853ac5d-654c-448a-be48-f8a394158d8e None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:53:56.521050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7853ac5d-654c-448a-be48-f8a394158d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:53:56.612963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 601/1207] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:53:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:53:56.638939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49ec0b-a9ee-44a8-b275-804263cc9dce None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:56.811599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49ec0b-a9ee-44a8-b275-804263cc9dce None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:56.878300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa82862-f80b-4af2-ab5e-1f2c42719f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:56.879334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa82862-f80b-4af2-ab5e-1f2c42719f65 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:56.879802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa82862-f80b-4af2-ab5e-1f2c42719f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:56.880264 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa82862-f80b-4af2-ab5e-1f2c42719f65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:56.893870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa82862-f80b-4af2-ab5e-1f2c42719f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-8aa82862-f80b-4af2-ab5e-1f2c42719f65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:53:56.899338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:53:56.900622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:53:56.901741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:53:56.904443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 607/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:53:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:53:56.910567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853ac5d-654c-448a-be48-f8a394158d8e None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:56.958144 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:57.076799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853ac5d-654c-448a-be48-f8a394158d8e None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:57.137701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaf004-bbf5-4e47-bbce-72076d809552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:57.138653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf004-bbf5-4e47-bbce-72076d809552 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:57.139108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf004-bbf5-4e47-bbce-72076d809552 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:57.139540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf004-bbf5-4e47-bbce-72076d809552 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:57.140497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfaf004-bbf5-4e47-bbce-72076d809552 None admin] RBAC: Adding query filter params (name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:53:57.140980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfaf004-bbf5-4e47-bbce-72076d809552 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:57.142605 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfaf004-bbf5-4e47-bbce-72076d809552 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:57.154507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 602/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:53:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:53:57.160748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa82862-f80b-4af2-ab5e-1f2c42719f65 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:57.225904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:57.292429 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa82862-f80b-4af2-ab5e-1f2c42719f65 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:57.336698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b2924-2c4d-4ab4-9414-a8730572d40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:57.337290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b2924-2c4d-4ab4-9414-a8730572d40e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:57.337573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b2924-2c4d-4ab4-9414-a8730572d40e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:57.337852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b2924-2c4d-4ab4-9414-a8730572d40e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:57.349686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b2924-2c4d-4ab4-9414-a8730572d40e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:57.351181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b2924-2c4d-4ab4-9414-a8730572d40e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-e75b2924-2c4d-4ab4-9414-a8730572d40e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:53:57.368651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:53:57.369869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:53:57.376495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:53:57.376495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:53:57.376495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:53:57.376495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:53:57.376495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 608/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:53:57.376843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaf004-bbf5-4e47-bbce-72076d809552 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:57.541079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:57.545271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaf004-bbf5-4e47-bbce-72076d809552 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:57.595819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3366e21-909f-4438-9737-a7f85a0319a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:57.596739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3366e21-909f-4438-9737-a7f85a0319a2 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:57.598991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3366e21-909f-4438-9737-a7f85a0319a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:57.598991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3366e21-909f-4438-9737-a7f85a0319a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:57.598991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3366e21-909f-4438-9737-a7f85a0319a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:53:57.598991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3366e21-909f-4438-9737-a7f85a0319a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:57.600728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3366e21-909f-4438-9737-a7f85a0319a2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:57.615655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 603/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:53:57.622178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b2924-2c4d-4ab4-9414-a8730572d40e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:57.778655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b2924-2c4d-4ab4-9414-a8730572d40e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:57.780752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:57.832388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f601c-5404-42b7-80e9-05b625e680d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:57.832388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f601c-5404-42b7-80e9-05b625e680d4 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:57.832388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f601c-5404-42b7-80e9-05b625e680d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:57.832388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f601c-5404-42b7-80e9-05b625e680d4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:57.840920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f601c-5404-42b7-80e9-05b625e680d4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:57.842497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f601c-5404-42b7-80e9-05b625e680d4 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-e44f601c-5404-42b7-80e9-05b625e680d4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:53:57.856828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:53:57.859190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:53:57.862103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 609/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:53:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:53:57.864815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3366e21-909f-4438-9737-a7f85a0319a2 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:58.013625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3366e21-909f-4438-9737-a7f85a0319a2 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:58.063572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa5ac1-2b41-462f-8954-b479d26cf51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:58.064267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa5ac1-2b41-462f-8954-b479d26cf51a None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:58.064631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa5ac1-2b41-462f-8954-b479d26cf51a None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:58.064877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa5ac1-2b41-462f-8954-b479d26cf51a None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:58.066705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa5ac1-2b41-462f-8954-b479d26cf51a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:53:58.067313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa5ac1-2b41-462f-8954-b479d26cf51a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:58.069042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa5ac1-2b41-462f-8954-b479d26cf51a None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:58.097868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 604/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Aug 1 11:53:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:53:58.108864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f601c-5404-42b7-80e9-05b625e680d4 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:58.282150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f601c-5404-42b7-80e9-05b625e680d4 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:58.335002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a94ab-cc17-4327-b004-d555b9269bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:58.335684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a94ab-cc17-4327-b004-d555b9269bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:58.336043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a94ab-cc17-4327-b004-d555b9269bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:58.336283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a94ab-cc17-4327-b004-d555b9269bc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:58.376459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:58.429007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a94ab-cc17-4327-b004-d555b9269bc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:58.430898 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a94ab-cc17-4327-b004-d555b9269bc9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:58.602743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 610/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:53:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:53:58.610779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa5ac1-2b41-462f-8954-b479d26cf51a None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:58.649653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:58.817261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa5ac1-2b41-462f-8954-b479d26cf51a None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:58.877880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937123cc-c00b-4652-8280-5e18735a3464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:58.878821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937123cc-c00b-4652-8280-5e18735a3464 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:58.879233 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937123cc-c00b-4652-8280-5e18735a3464 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:58.879523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937123cc-c00b-4652-8280-5e18735a3464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:58.894726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937123cc-c00b-4652-8280-5e18735a3464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:58.896580 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937123cc-c00b-4652-8280-5e18735a3464 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-937123cc-c00b-4652-8280-5e18735a3464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:53:58.915892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:53:58.917905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:53:58.920017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:53:58.920017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:53:58.920017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:53:58.920017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:53:58.922097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 605/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:53:58.924576 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a94ab-cc17-4327-b004-d555b9269bc9 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:59.048247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:59.115654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a94ab-cc17-4327-b004-d555b9269bc9 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:59.187066 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fe03f-66c5-44a2-a2f8-c09f98a2c39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:59.187636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fe03f-66c5-44a2-a2f8-c09f98a2c39d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:59.187923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fe03f-66c5-44a2-a2f8-c09f98a2c39d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:59.188223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fe03f-66c5-44a2-a2f8-c09f98a2c39d None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:59.189057 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fe03f-66c5-44a2-a2f8-c09f98a2c39d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:53:59.189399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fe03f-66c5-44a2-a2f8-c09f98a2c39d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:59.190991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894fe03f-66c5-44a2-a2f8-c09f98a2c39d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:59.206411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 611/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:53:59.214169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937123cc-c00b-4652-8280-5e18735a3464 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:59.338152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:59.398449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937123cc-c00b-4652-8280-5e18735a3464 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:59.451149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f76ab0-65a5-427f-b801-227d0379b719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:59.451978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f76ab0-65a5-427f-b801-227d0379b719 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:59.452425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f76ab0-65a5-427f-b801-227d0379b719 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:59.452583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f76ab0-65a5-427f-b801-227d0379b719 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:59.461933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f76ab0-65a5-427f-b801-227d0379b719 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:59.463579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f76ab0-65a5-427f-b801-227d0379b719 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-e9f76ab0-65a5-427f-b801-227d0379b719 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:53:59.479754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:53:59.481353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:53:59.483315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:53:59.483315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:53:59.483315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:53:59.483315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:53:59.483315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:53:59.483315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:53:59.483315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:53:59.483315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:53:59.483315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 606/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:53:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:53:59.488366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fe03f-66c5-44a2-a2f8-c09f98a2c39d None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:59.629899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fe03f-66c5-44a2-a2f8-c09f98a2c39d None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:59.658731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:59.676586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b332d-c4be-4d1f-b4cf-73570bec5ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:59.677400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b332d-c4be-4d1f-b4cf-73570bec5ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:59.677684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b332d-c4be-4d1f-b4cf-73570bec5ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:59.678009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b332d-c4be-4d1f-b4cf-73570bec5ee7 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:59.679003 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b332d-c4be-4d1f-b4cf-73570bec5ee7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:53:59.679304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b332d-c4be-4d1f-b4cf-73570bec5ee7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:59.681041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b332d-c4be-4d1f-b4cf-73570bec5ee7 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:59.697417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 612/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:53:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:53:59.705550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f76ab0-65a5-427f-b801-227d0379b719 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:53:59.856623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f76ab0-65a5-427f-b801-227d0379b719 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:53:59.892466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:53:59.899707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8db675-ddf6-431c-b919-536c77c0fa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:53:59.901206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8db675-ddf6-431c-b919-536c77c0fa71 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:53:59.901588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8db675-ddf6-431c-b919-536c77c0fa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:53:59.902008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8db675-ddf6-431c-b919-536c77c0fa71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:53:59.902921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8db675-ddf6-431c-b919-536c77c0fa71 None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=9db663d35f964d42938be0d1a9ed4071) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:53:59.903338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8db675-ddf6-431c-b919-536c77c0fa71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:53:59.904930 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8db675-ddf6-431c-b919-536c77c0fa71 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:53:59.913937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 607/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:53:59 2022] GET /identity/v3/role_assignments?user.id=9db663d35f964d42938be0d1a9ed4071&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:02.493396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac40c9-c8e0-4342-a79f-234b727706bc None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:02.494038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac40c9-c8e0-4342-a79f-234b727706bc None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:02.494296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac40c9-c8e0-4342-a79f-234b727706bc None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:02.497068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 613/1220] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:54:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:54:02.506407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29705af3-ddca-448f-a743-5cfdcf436c20 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:02.507280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29705af3-ddca-448f-a743-5cfdcf436c20 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:02.507455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29705af3-ddca-448f-a743-5cfdcf436c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:02.535864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-29705af3-ddca-448f-a743-5cfdcf436c20 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:54:02.537997 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-29705af3-ddca-448f-a743-5cfdcf436c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:54:02.650457 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 608/1221] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:54:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:54:02.673879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac40c9-c8e0-4342-a79f-234b727706bc None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:02.857020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac40c9-c8e0-4342-a79f-234b727706bc None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:02.901999 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73178c0-b36d-42dc-8080-4d95c6ac6eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:02.902706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73178c0-b36d-42dc-8080-4d95c6ac6eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:02.903033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73178c0-b36d-42dc-8080-4d95c6ac6eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:02.903364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73178c0-b36d-42dc-8080-4d95c6ac6eb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:02.916484 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73178c0-b36d-42dc-8080-4d95c6ac6eb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-f73178c0-b36d-42dc-8080-4d95c6ac6eb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:54:02.922129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:02.924078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:02.925245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 614/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:54:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:54:02.933929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29705af3-ddca-448f-a743-5cfdcf436c20 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:03.095523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29705af3-ddca-448f-a743-5cfdcf436c20 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:03.131443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:03.151938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1bd43-6d3c-452c-b1c3-a1befefe232b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:03.153293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1bd43-6d3c-452c-b1c3-a1befefe232b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:03.153945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1bd43-6d3c-452c-b1c3-a1befefe232b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:03.154646 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1bd43-6d3c-452c-b1c3-a1befefe232b None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:03.157813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1bd43-6d3c-452c-b1c3-a1befefe232b None admin] RBAC: Adding query filter params (name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:03.158079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1bd43-6d3c-452c-b1c3-a1befefe232b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:03.161031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1bd43-6d3c-452c-b1c3-a1befefe232b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:03.173913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 609/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:54:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:03.180435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73178c0-b36d-42dc-8080-4d95c6ac6eb3 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:03.261466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:03.362107 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73178c0-b36d-42dc-8080-4d95c6ac6eb3 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:03.416336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5944f-02b1-415c-b281-3dcecb73a154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:03.417077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5944f-02b1-415c-b281-3dcecb73a154 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:03.417520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5944f-02b1-415c-b281-3dcecb73a154 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:03.417974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f5944f-02b1-415c-b281-3dcecb73a154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:03.437557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5944f-02b1-415c-b281-3dcecb73a154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:03.439976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f5944f-02b1-415c-b281-3dcecb73a154 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-01f5944f-02b1-415c-b281-3dcecb73a154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:54:03.467963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:03.470082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:03.471424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:54:03.471424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:54:03.471424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:03.471424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:03.471424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 615/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:03.478494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1bd43-6d3c-452c-b1c3-a1befefe232b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:03.653134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1bd43-6d3c-452c-b1c3-a1befefe232b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:03.714893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a0c62-9e39-4a0b-b699-89771d344a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:03.715834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a0c62-9e39-4a0b-b699-89771d344a8e None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:03.716338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a0c62-9e39-4a0b-b699-89771d344a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:03.716821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a0c62-9e39-4a0b-b699-89771d344a8e None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:03.718131 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a0c62-9e39-4a0b-b699-89771d344a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:03.718623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a0c62-9e39-4a0b-b699-89771d344a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:03.721193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a0c62-9e39-4a0b-b699-89771d344a8e None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:03.742786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 610/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:03.751096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5944f-02b1-415c-b281-3dcecb73a154 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:03.952541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:03.962740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f5944f-02b1-415c-b281-3dcecb73a154 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:04.024374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f3d4f-a691-4628-b715-bb5b5222c208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:04.025943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f3d4f-a691-4628-b715-bb5b5222c208 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:04.026557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f3d4f-a691-4628-b715-bb5b5222c208 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:04.027046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f3d4f-a691-4628-b715-bb5b5222c208 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:04.044183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f3d4f-a691-4628-b715-bb5b5222c208 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:04.047628 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018f3d4f-a691-4628-b715-bb5b5222c208 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-018f3d4f-a691-4628-b715-bb5b5222c208 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:54:04.071274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:04.073323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:04.075100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 616/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:54:04.083096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a0c62-9e39-4a0b-b699-89771d344a8e None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:04.171643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:04.279945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a0c62-9e39-4a0b-b699-89771d344a8e None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:04.342873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3da17-6c14-41f9-a9b3-3e7148315d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:04.343669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3da17-6c14-41f9-a9b3-3e7148315d0b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:04.344032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3da17-6c14-41f9-a9b3-3e7148315d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:04.344444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3da17-6c14-41f9-a9b3-3e7148315d0b None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:04.345397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3da17-6c14-41f9-a9b3-3e7148315d0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:04.345784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3da17-6c14-41f9-a9b3-3e7148315d0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:04.347350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3da17-6c14-41f9-a9b3-3e7148315d0b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:04.372281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 611/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Aug 1 11:54:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:04.379833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f3d4f-a691-4628-b715-bb5b5222c208 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:04.547636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f3d4f-a691-4628-b715-bb5b5222c208 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:04.598388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ab97d-30b6-4e17-8fc4-e4d187f2bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:04.599086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ab97d-30b6-4e17-8fc4-e4d187f2bc12 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:04.599349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ab97d-30b6-4e17-8fc4-e4d187f2bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:04.599662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ab97d-30b6-4e17-8fc4-e4d187f2bc12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:04.616072 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ab97d-30b6-4e17-8fc4-e4d187f2bc12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:04.618821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ab97d-30b6-4e17-8fc4-e4d187f2bc12 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-416ab97d-30b6-4e17-8fc4-e4d187f2bc12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:54:04.645123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:04.646300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:04.647413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:54:04.647413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:54:04.647413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:04.647413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:04.647413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 617/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:04.656638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3da17-6c14-41f9-a9b3-3e7148315d0b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:04.796104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3da17-6c14-41f9-a9b3-3e7148315d0b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:04.799327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:04.843236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58c1bf-e8c7-414c-a44c-305bd16b6e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:04.843943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58c1bf-e8c7-414c-a44c-305bd16b6e71 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:04.844251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58c1bf-e8c7-414c-a44c-305bd16b6e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:04.844597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58c1bf-e8c7-414c-a44c-305bd16b6e71 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:04.845357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58c1bf-e8c7-414c-a44c-305bd16b6e71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:04.845690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58c1bf-e8c7-414c-a44c-305bd16b6e71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:04.847203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b58c1bf-e8c7-414c-a44c-305bd16b6e71 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:04.862258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 612/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:04.868883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ab97d-30b6-4e17-8fc4-e4d187f2bc12 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:04.882286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:05.010151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ab97d-30b6-4e17-8fc4-e4d187f2bc12 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:05.056540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f85cdb-c2ac-4980-ad66-a5e92641dacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:05.057288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f85cdb-c2ac-4980-ad66-a5e92641dacb None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:05.057588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f85cdb-c2ac-4980-ad66-a5e92641dacb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:05.057901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f85cdb-c2ac-4980-ad66-a5e92641dacb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:05.066975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f85cdb-c2ac-4980-ad66-a5e92641dacb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:05.068768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f85cdb-c2ac-4980-ad66-a5e92641dacb None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-f7f85cdb-c2ac-4980-ad66-a5e92641dacb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:54:05.087372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:05.089390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:05.091405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:54:05.091405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:54:05.091405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:54:05.091405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:54:05.091405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:54:05.091405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:54:05.091405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:54:05.091405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:05.091405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 618/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:54:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:05.096135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58c1bf-e8c7-414c-a44c-305bd16b6e71 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:05.234021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58c1bf-e8c7-414c-a44c-305bd16b6e71 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:05.294010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b5bcd-ff7a-4c33-8f20-ca03c4477474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:05.294985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5bcd-ff7a-4c33-8f20-ca03c4477474 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:05.297273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5bcd-ff7a-4c33-8f20-ca03c4477474 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:05.297273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5bcd-ff7a-4c33-8f20-ca03c4477474 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:05.297273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b5bcd-ff7a-4c33-8f20-ca03c4477474 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:05.297273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b5bcd-ff7a-4c33-8f20-ca03c4477474 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:05.299683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b5bcd-ff7a-4c33-8f20-ca03c4477474 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:05.320586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 613/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:54:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:54:05.327493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f85cdb-c2ac-4980-ad66-a5e92641dacb None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:05.441917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:05.505150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f85cdb-c2ac-4980-ad66-a5e92641dacb None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:05.565683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36672fb-06dd-4c00-9f08-8348697b95bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:05.566569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36672fb-06dd-4c00-9f08-8348697b95bb None admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:05.566887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36672fb-06dd-4c00-9f08-8348697b95bb None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:05.567256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36672fb-06dd-4c00-9f08-8348697b95bb None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/9db663d35f964d42938be0d1a9ed4071/roles/a55004b069fa4cd99327dd3d4f55efcf` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:05.589093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36672fb-06dd-4c00-9f08-8348697b95bb None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=9db663d35f964d42938be0d1a9ed4071, role_id=a55004b069fa4cd99327dd3d4f55efcf)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:05.590858 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36672fb-06dd-4c00-9f08-8348697b95bb None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:05.609083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 619/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:54:05 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/9db663d35f964d42938be0d1a9ed4071/roles/a55004b069fa4cd99327dd3d4f55efcf => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 11:54:08.028974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef10a5-e7fe-4e59-9bfd-81fc38147080 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:08.028974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef10a5-e7fe-4e59-9bfd-81fc38147080 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:08.028974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef10a5-e7fe-4e59-9bfd-81fc38147080 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:08.028974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 614/1233] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:54:08.035523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318aa39e-c3db-46ae-a126-580ba3badbae None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:08.036256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318aa39e-c3db-46ae-a126-580ba3badbae None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:08.036497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318aa39e-c3db-46ae-a126-580ba3badbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:08.068627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-318aa39e-c3db-46ae-a126-580ba3badbae None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:54:08.070801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-318aa39e-c3db-46ae-a126-580ba3badbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:54:08.201383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 620/1234] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:54:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:54:08.228884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef10a5-e7fe-4e59-9bfd-81fc38147080 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:08.264353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:08.377639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef10a5-e7fe-4e59-9bfd-81fc38147080 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:08.435816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e1154-369c-4d39-a25d-2ca150ee2ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:08.436738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e1154-369c-4d39-a25d-2ca150ee2ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:08.437112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e1154-369c-4d39-a25d-2ca150ee2ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:08.437425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5e1154-369c-4d39-a25d-2ca150ee2ab6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:08.453932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5e1154-369c-4d39-a25d-2ca150ee2ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-4e5e1154-369c-4d39-a25d-2ca150ee2ab6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:54:08.459917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:08.461551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:08.463152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 615/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Aug 1 11:54:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:08.470320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318aa39e-c3db-46ae-a126-580ba3badbae None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:08.539774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:08.622830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318aa39e-c3db-46ae-a126-580ba3badbae None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:08.673046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acade42-b73d-42bb-bb06-390986124438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:08.674128 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acade42-b73d-42bb-bb06-390986124438 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:08.674608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acade42-b73d-42bb-bb06-390986124438 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:08.675135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acade42-b73d-42bb-bb06-390986124438 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:08.676482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acade42-b73d-42bb-bb06-390986124438 None admin] RBAC: Adding query filter params (name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:08.676983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acade42-b73d-42bb-bb06-390986124438 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:08.679517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acade42-b73d-42bb-bb06-390986124438 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:08.702894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 621/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Aug 1 11:54:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:08.704728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e1154-369c-4d39-a25d-2ca150ee2ab6 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:08.839571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:08.864919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5e1154-369c-4d39-a25d-2ca150ee2ab6 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:08.910400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66054f-acda-40bc-93d2-d38667e24120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:08.911213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66054f-acda-40bc-93d2-d38667e24120 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:08.911668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66054f-acda-40bc-93d2-d38667e24120 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:08.912100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66054f-acda-40bc-93d2-d38667e24120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:08.925251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66054f-acda-40bc-93d2-d38667e24120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:08.926736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66054f-acda-40bc-93d2-d38667e24120 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-db66054f-acda-40bc-93d2-d38667e24120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:54:08.943483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:08.953654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:08.955598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:54:08.955598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:54:08.955598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:08.955598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:08.955598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 616/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:08.956230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acade42-b73d-42bb-bb06-390986124438 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:09.108568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acade42-b73d-42bb-bb06-390986124438 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:09.127327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:09.175343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4349564-ef59-4d69-a0bd-20389f31e570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:09.176144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4349564-ef59-4d69-a0bd-20389f31e570 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:09.176589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4349564-ef59-4d69-a0bd-20389f31e570 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:09.176977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4349564-ef59-4d69-a0bd-20389f31e570 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:09.177796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4349564-ef59-4d69-a0bd-20389f31e570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:09.178197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4349564-ef59-4d69-a0bd-20389f31e570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:09.179695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4349564-ef59-4d69-a0bd-20389f31e570 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:09.204620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 622/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:54:09.211664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66054f-acda-40bc-93d2-d38667e24120 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:09.350783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66054f-acda-40bc-93d2-d38667e24120 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:09.393262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a038a-7ed6-4565-8397-ae64af36ef5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:09.393923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a038a-7ed6-4565-8397-ae64af36ef5f None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:09.394210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a038a-7ed6-4565-8397-ae64af36ef5f None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:09.394493 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a038a-7ed6-4565-8397-ae64af36ef5f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:09.398479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:09.405309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a038a-7ed6-4565-8397-ae64af36ef5f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:09.407072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a038a-7ed6-4565-8397-ae64af36ef5f None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-e27a038a-7ed6-4565-8397-ae64af36ef5f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:54:09.420009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:09.422342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:09.424379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 617/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:09.427754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4349564-ef59-4d69-a0bd-20389f31e570 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:09.616610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4349564-ef59-4d69-a0bd-20389f31e570 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:09.673311 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbe6c2-c1e8-4cf8-8602-2b4f4f7ad916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:09.673976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbe6c2-c1e8-4cf8-8602-2b4f4f7ad916 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:09.674257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbe6c2-c1e8-4cf8-8602-2b4f4f7ad916 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:09.674533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbe6c2-c1e8-4cf8-8602-2b4f4f7ad916 None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:09.675368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbe6c2-c1e8-4cf8-8602-2b4f4f7ad916 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:09.675676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbe6c2-c1e8-4cf8-8602-2b4f4f7ad916 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:09.677249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbe6c2-c1e8-4cf8-8602-2b4f4f7ad916 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:09.701328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 623/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Aug 1 11:54:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:09.707822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a038a-7ed6-4565-8397-ae64af36ef5f None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:09.849025 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a038a-7ed6-4565-8397-ae64af36ef5f None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:09.900714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb66eb-803c-41ef-9f06-a3e328f70e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:09.901534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb66eb-803c-41ef-9f06-a3e328f70e04 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:09.901852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb66eb-803c-41ef-9f06-a3e328f70e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:09.902135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb66eb-803c-41ef-9f06-a3e328f70e04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:10.003064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:10.014838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb66eb-803c-41ef-9f06-a3e328f70e04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:10.016525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb66eb-803c-41ef-9f06-a3e328f70e04 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:10.142773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 618/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:54:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 11:54:10.149528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbe6c2-c1e8-4cf8-8602-2b4f4f7ad916 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:10.189498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:10.279517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbe6c2-c1e8-4cf8-8602-2b4f4f7ad916 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:10.330419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425215e-e4ed-4a88-b469-7d7b8f90324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:10.331100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425215e-e4ed-4a88-b469-7d7b8f90324b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:10.331381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425215e-e4ed-4a88-b469-7d7b8f90324b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:10.331661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425215e-e4ed-4a88-b469-7d7b8f90324b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:10.344326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425215e-e4ed-4a88-b469-7d7b8f90324b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:10.346011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425215e-e4ed-4a88-b469-7d7b8f90324b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-5425215e-e4ed-4a88-b469-7d7b8f90324b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:54:10.365157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:10.366995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:10.368073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:54:10.368073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:54:10.368073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:10.368073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:10.368073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 624/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:10.376753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb66eb-803c-41ef-9f06-a3e328f70e04 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:10.510143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:10.522538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb66eb-803c-41ef-9f06-a3e328f70e04 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:10.576967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48865e3-c3f6-42a3-a1bb-5004f2c3954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:10.577624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48865e3-c3f6-42a3-a1bb-5004f2c3954d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:10.577923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48865e3-c3f6-42a3-a1bb-5004f2c3954d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:10.578191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48865e3-c3f6-42a3-a1bb-5004f2c3954d None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:10.578916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48865e3-c3f6-42a3-a1bb-5004f2c3954d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:10.579217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48865e3-c3f6-42a3-a1bb-5004f2c3954d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:10.580683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48865e3-c3f6-42a3-a1bb-5004f2c3954d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:10.609250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 619/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:10.609488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425215e-e4ed-4a88-b469-7d7b8f90324b None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:10.703540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:10.731632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425215e-e4ed-4a88-b469-7d7b8f90324b None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:10.785900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a64be-676c-4387-8309-325019516522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:10.786577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a64be-676c-4387-8309-325019516522 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:10.786967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a64be-676c-4387-8309-325019516522 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:10.787322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a64be-676c-4387-8309-325019516522 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:10.797213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a64be-676c-4387-8309-325019516522 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:10.799371 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430a64be-676c-4387-8309-325019516522 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-430a64be-676c-4387-8309-325019516522 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:54:10.814862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:10.816662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:10.817728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:54:10.817728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:54:10.817728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:54:10.817728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:54:10.817728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:54:10.817728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:54:10.817728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:54:10.817728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:10.817728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 625/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:54:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:10.822958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48865e3-c3f6-42a3-a1bb-5004f2c3954d None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:10.965088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48865e3-c3f6-42a3-a1bb-5004f2c3954d None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:11.008139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7157baa5-d3f3-4577-ba87-376db899d2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:11.008893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157baa5-d3f3-4577-ba87-376db899d2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:11.009176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157baa5-d3f3-4577-ba87-376db899d2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:11.009454 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7157baa5-d3f3-4577-ba87-376db899d2b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:11.010298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157baa5-d3f3-4577-ba87-376db899d2b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:11.010625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157baa5-d3f3-4577-ba87-376db899d2b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:11.012206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7157baa5-d3f3-4577-ba87-376db899d2b2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:11.028151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 620/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:54:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:11.034978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a64be-676c-4387-8309-325019516522 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:11.172578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430a64be-676c-4387-8309-325019516522 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:11.214423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c21572-9d38-4837-92ac-d4b22ff2eee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:11.215150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c21572-9d38-4837-92ac-d4b22ff2eee9 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:11.215451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c21572-9d38-4837-92ac-d4b22ff2eee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:11.215703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c21572-9d38-4837-92ac-d4b22ff2eee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:11.216629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c21572-9d38-4837-92ac-d4b22ff2eee9 None admin] RBAC: Adding query filter params (role.id=a55004b069fa4cd99327dd3d4f55efcf, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=9db663d35f964d42938be0d1a9ed4071) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:11.216946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c21572-9d38-4837-92ac-d4b22ff2eee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:11.218517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c21572-9d38-4837-92ac-d4b22ff2eee9 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:11.225933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 626/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:54:11 2022] GET /identity/v3/role_assignments?user.id=9db663d35f964d42938be0d1a9ed4071&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=a55004b069fa4cd99327dd3d4f55efcf => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:54:13.722358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda33728-1fd0-48bd-988c-449174d5b8cf None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:13.723048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda33728-1fd0-48bd-988c-449174d5b8cf None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:13.723606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda33728-1fd0-48bd-988c-449174d5b8cf None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:13.726665 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 621/1247] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:54:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:54:13.734328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac941f8-9cee-4526-a2aa-ce714c6b90ec None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:13.735186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac941f8-9cee-4526-a2aa-ce714c6b90ec None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:13.735186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac941f8-9cee-4526-a2aa-ce714c6b90ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:13.745409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:13.762857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-cac941f8-9cee-4526-a2aa-ce714c6b90ec None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:54:13.764656 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-cac941f8-9cee-4526-a2aa-ce714c6b90ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:54:13.860640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 627/1248] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:54:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:54:13.886932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda33728-1fd0-48bd-988c-449174d5b8cf None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:13.928190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:14.078379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda33728-1fd0-48bd-988c-449174d5b8cf None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:14.131245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc289d27-ccd3-48b5-94cb-3f4854852284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:14.131989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc289d27-ccd3-48b5-94cb-3f4854852284 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:14.132365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc289d27-ccd3-48b5-94cb-3f4854852284 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:14.132772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc289d27-ccd3-48b5-94cb-3f4854852284 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:14.152883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc289d27-ccd3-48b5-94cb-3f4854852284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-fc289d27-ccd3-48b5-94cb-3f4854852284 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:54:14.160009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:14.162062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:14.163690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 622/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:54:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:14.171310 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac941f8-9cee-4526-a2aa-ce714c6b90ec None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:14.313359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac941f8-9cee-4526-a2aa-ce714c6b90ec None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:14.368976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:14.374521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da43e6d-3e3d-4fa8-b478-2d81960a901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:14.375477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da43e6d-3e3d-4fa8-b478-2d81960a901a None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:14.375781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da43e6d-3e3d-4fa8-b478-2d81960a901a None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:14.376223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da43e6d-3e3d-4fa8-b478-2d81960a901a None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:14.377138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da43e6d-3e3d-4fa8-b478-2d81960a901a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:14.377891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da43e6d-3e3d-4fa8-b478-2d81960a901a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:14.380974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da43e6d-3e3d-4fa8-b478-2d81960a901a None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:14.393199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 628/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:54:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:14.401860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc289d27-ccd3-48b5-94cb-3f4854852284 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:14.547198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:14.586377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc289d27-ccd3-48b5-94cb-3f4854852284 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:14.635312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d029-ac2b-4298-9a04-2cd5ed527612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:14.635994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d029-ac2b-4298-9a04-2cd5ed527612 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:14.636386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d029-ac2b-4298-9a04-2cd5ed527612 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:14.636859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e2d029-ac2b-4298-9a04-2cd5ed527612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:14.652044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2d029-ac2b-4298-9a04-2cd5ed527612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:14.654174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e2d029-ac2b-4298-9a04-2cd5ed527612 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-49e2d029-ac2b-4298-9a04-2cd5ed527612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:54:14.677299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:14.679018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:14.680667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:54:14.680667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:54:14.680667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:14.680667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:14.680667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 623/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:14.687089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da43e6d-3e3d-4fa8-b478-2d81960a901a None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:14.842932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da43e6d-3e3d-4fa8-b478-2d81960a901a None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:14.900027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79600625-ec8a-440e-a5dd-29ccc8a4ce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:14.900658 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79600625-ec8a-440e-a5dd-29ccc8a4ce62 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:14.900943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79600625-ec8a-440e-a5dd-29ccc8a4ce62 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:14.901279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79600625-ec8a-440e-a5dd-29ccc8a4ce62 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:14.902008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79600625-ec8a-440e-a5dd-29ccc8a4ce62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:14.902315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79600625-ec8a-440e-a5dd-29ccc8a4ce62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:14.903861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79600625-ec8a-440e-a5dd-29ccc8a4ce62 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:14.925986 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 629/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:14.934480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d029-ac2b-4298-9a04-2cd5ed527612 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:15.104449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e2d029-ac2b-4298-9a04-2cd5ed527612 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:15.151335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:15.163769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef87d18-ad90-4196-bff2-c12944444d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:15.164851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef87d18-ad90-4196-bff2-c12944444d52 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:15.165230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef87d18-ad90-4196-bff2-c12944444d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:15.165612 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef87d18-ad90-4196-bff2-c12944444d52 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:15.177365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef87d18-ad90-4196-bff2-c12944444d52 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:15.179311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef87d18-ad90-4196-bff2-c12944444d52 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-6ef87d18-ad90-4196-bff2-c12944444d52 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:54:15.195266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:15.197387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:15.198862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 624/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:54:15.204921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79600625-ec8a-440e-a5dd-29ccc8a4ce62 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:15.297673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:15.385346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79600625-ec8a-440e-a5dd-29ccc8a4ce62 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:15.463794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89c641-29d9-444a-b191-b45df3d5cfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:15.465340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89c641-29d9-444a-b191-b45df3d5cfdf None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:15.466035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89c641-29d9-444a-b191-b45df3d5cfdf None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:15.466523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89c641-29d9-444a-b191-b45df3d5cfdf None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:15.468627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89c641-29d9-444a-b191-b45df3d5cfdf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:15.469321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89c641-29d9-444a-b191-b45df3d5cfdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:15.472907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89c641-29d9-444a-b191-b45df3d5cfdf None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:15.500956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 630/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Aug 1 11:54:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:54:15.511491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef87d18-ad90-4196-bff2-c12944444d52 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:15.653219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef87d18-ad90-4196-bff2-c12944444d52 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:15.702624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8db0eb-7b10-4683-8686-7ad876693a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:15.703297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db0eb-7b10-4683-8686-7ad876693a88 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:15.703735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db0eb-7b10-4683-8686-7ad876693a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:15.704124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8db0eb-7b10-4683-8686-7ad876693a88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:15.786950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:15.812477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8db0eb-7b10-4683-8686-7ad876693a88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:15.815505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8db0eb-7b10-4683-8686-7ad876693a88 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:15.980781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 625/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:54:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 01 11:54:15.988058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89c641-29d9-444a-b191-b45df3d5cfdf None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:16.157490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89c641-29d9-444a-b191-b45df3d5cfdf None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:16.179110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:16.210709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f780b48-2167-4349-ac97-dae5f61e3dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:16.211775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f780b48-2167-4349-ac97-dae5f61e3dba None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:16.212396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f780b48-2167-4349-ac97-dae5f61e3dba None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:16.212917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f780b48-2167-4349-ac97-dae5f61e3dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:16.234852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f780b48-2167-4349-ac97-dae5f61e3dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:16.236687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f780b48-2167-4349-ac97-dae5f61e3dba None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-2f780b48-2167-4349-ac97-dae5f61e3dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:54:16.258948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:16.261030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:16.262868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:54:16.262868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:54:16.262868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:16.262868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:16.262868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 631/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:16.267984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8db0eb-7b10-4683-8686-7ad876693a88 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:16.412626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:16.432667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8db0eb-7b10-4683-8686-7ad876693a88 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:16.496511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef78da7-8edf-4e0d-a244-090f7d31614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:16.497533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef78da7-8edf-4e0d-a244-090f7d31614c None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:16.498038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef78da7-8edf-4e0d-a244-090f7d31614c None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:16.500000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef78da7-8edf-4e0d-a244-090f7d31614c None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:16.501387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef78da7-8edf-4e0d-a244-090f7d31614c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:16.501883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef78da7-8edf-4e0d-a244-090f7d31614c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:16.504529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef78da7-8edf-4e0d-a244-090f7d31614c None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:16.535259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 626/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:54:16.537179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f780b48-2167-4349-ac97-dae5f61e3dba None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:16.685593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f780b48-2167-4349-ac97-dae5f61e3dba None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:16.731725 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fb61e-7a4b-4970-8d83-22c75e99b2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:16.732478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fb61e-7a4b-4970-8d83-22c75e99b2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:16.732754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fb61e-7a4b-4970-8d83-22c75e99b2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:16.733019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fb61e-7a4b-4970-8d83-22c75e99b2c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:16.741686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fb61e-7a4b-4970-8d83-22c75e99b2c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:16.743258 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fb61e-7a4b-4970-8d83-22c75e99b2c3 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-488fb61e-7a4b-4970-8d83-22c75e99b2c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:54:16.759455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:16.760984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:16.762073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:54:16.762073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:54:16.762073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:54:16.762073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:54:16.762073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:54:16.762073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:54:16.762073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:54:16.762073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:16.762073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 632/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:16.769767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef78da7-8edf-4e0d-a244-090f7d31614c None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:16.917804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef78da7-8edf-4e0d-a244-090f7d31614c None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:16.972645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34b333-0123-4d54-8f63-647de99e1ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:16.973251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34b333-0123-4d54-8f63-647de99e1ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:16.973569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34b333-0123-4d54-8f63-647de99e1ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:16.973865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34b333-0123-4d54-8f63-647de99e1ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:16.974646 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34b333-0123-4d54-8f63-647de99e1ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:16.974957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34b333-0123-4d54-8f63-647de99e1ee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:16.976927 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34b333-0123-4d54-8f63-647de99e1ee1 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:16.993303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 627/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:17.002534 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fb61e-7a4b-4970-8d83-22c75e99b2c3 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:17.048514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:17.184436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fb61e-7a4b-4970-8d83-22c75e99b2c3 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:17.246267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5153f6-de2b-4fb3-9ce3-a978be172e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:17.247053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5153f6-de2b-4fb3-9ce3-a978be172e0e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:17.247665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5153f6-de2b-4fb3-9ce3-a978be172e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:17.248131 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5153f6-de2b-4fb3-9ce3-a978be172e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:17.250068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5153f6-de2b-4fb3-9ce3-a978be172e0e None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=9db663d35f964d42938be0d1a9ed4071) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:17.250624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5153f6-de2b-4fb3-9ce3-a978be172e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:17.253644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5153f6-de2b-4fb3-9ce3-a978be172e0e None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:17.268333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 633/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:54:17 2022] GET /identity/v3/role_assignments?user.id=9db663d35f964d42938be0d1a9ed4071&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:19.848807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27537c1a-ea03-40e4-b1f3-0978d168e9b4 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:19.849703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27537c1a-ea03-40e4-b1f3-0978d168e9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:19.849703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27537c1a-ea03-40e4-b1f3-0978d168e9b4 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:19.851648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 628/1261] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:54:19.858007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfab11-f03b-45b2-b342-9506ac105061 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:19.858728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfab11-f03b-45b2-b342-9506ac105061 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:19.858728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfab11-f03b-45b2-b342-9506ac105061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:19.879671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-61bfab11-f03b-45b2-b342-9506ac105061 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:54:19.880996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-61bfab11-f03b-45b2-b342-9506ac105061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:54:19.974507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 634/1262] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:54:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:54:20.003472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27537c1a-ea03-40e4-b1f3-0978d168e9b4 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:20.017352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:20.149756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27537c1a-ea03-40e4-b1f3-0978d168e9b4 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:20.201066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b11fe-4bf4-4e5b-bab4-ed081a35263b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:20.201783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b11fe-4bf4-4e5b-bab4-ed081a35263b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:20.202064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b11fe-4bf4-4e5b-bab4-ed081a35263b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:20.202353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b11fe-4bf4-4e5b-bab4-ed081a35263b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:20.216134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b11fe-4bf4-4e5b-bab4-ed081a35263b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-538b11fe-4bf4-4e5b-bab4-ed081a35263b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:54:20.221887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:20.223913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:20.224952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 629/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:54:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:20.230877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfab11-f03b-45b2-b342-9506ac105061 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:20.254715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:20.361397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfab11-f03b-45b2-b342-9506ac105061 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:20.405648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99c315-361a-4f1c-a744-278a7687e4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:20.406279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99c315-361a-4f1c-a744-278a7687e4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:20.406610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99c315-361a-4f1c-a744-278a7687e4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:20.406967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99c315-361a-4f1c-a744-278a7687e4c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:20.408101 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99c315-361a-4f1c-a744-278a7687e4c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:20.408590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99c315-361a-4f1c-a744-278a7687e4c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:20.410124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99c315-361a-4f1c-a744-278a7687e4c2 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:20.421842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 635/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:54:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:20.430169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b11fe-4bf4-4e5b-bab4-ed081a35263b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:20.550346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:20.601294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b11fe-4bf4-4e5b-bab4-ed081a35263b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:20.650079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d600d-9a22-4f9a-af33-a1282f3a97e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:20.651251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d600d-9a22-4f9a-af33-a1282f3a97e2 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:20.651825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d600d-9a22-4f9a-af33-a1282f3a97e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:20.652389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d600d-9a22-4f9a-af33-a1282f3a97e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:20.673543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d600d-9a22-4f9a-af33-a1282f3a97e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:20.676397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d600d-9a22-4f9a-af33-a1282f3a97e2 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-918d600d-9a22-4f9a-af33-a1282f3a97e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:54:20.706310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:20.707516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:20.708629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:54:20.708629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:54:20.708629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:20.708629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:20.708629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 630/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:54:20.714837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99c315-361a-4f1c-a744-278a7687e4c2 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:20.810295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:20.834059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99c315-361a-4f1c-a744-278a7687e4c2 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:20.876962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7397ac0-bd78-4dce-bfb3-ecaa27bc9fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:20.877749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7397ac0-bd78-4dce-bfb3-ecaa27bc9fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:20.877885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7397ac0-bd78-4dce-bfb3-ecaa27bc9fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:20.878136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7397ac0-bd78-4dce-bfb3-ecaa27bc9fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:20.878859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7397ac0-bd78-4dce-bfb3-ecaa27bc9fd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:20.879168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7397ac0-bd78-4dce-bfb3-ecaa27bc9fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:20.880650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7397ac0-bd78-4dce-bfb3-ecaa27bc9fd7 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:20.894470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 636/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:20.901439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d600d-9a22-4f9a-af33-a1282f3a97e2 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:21.023709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d600d-9a22-4f9a-af33-a1282f3a97e2 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:21.047987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:21.065064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb958c5-4121-42c3-9734-444ccc0de190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:21.065742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb958c5-4121-42c3-9734-444ccc0de190 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:21.066020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb958c5-4121-42c3-9734-444ccc0de190 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:21.066288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb958c5-4121-42c3-9734-444ccc0de190 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:21.076211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb958c5-4121-42c3-9734-444ccc0de190 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:21.077749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb958c5-4121-42c3-9734-444ccc0de190 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-fbb958c5-4121-42c3-9734-444ccc0de190 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:54:21.090452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:21.092416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:21.094360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 631/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:21.101692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7397ac0-bd78-4dce-bfb3-ecaa27bc9fd7 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:21.291178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7397ac0-bd78-4dce-bfb3-ecaa27bc9fd7 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:21.355339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c4f65-a7ba-4841-bd8f-9627831381fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:21.356411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c4f65-a7ba-4841-bd8f-9627831381fa None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:21.356899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c4f65-a7ba-4841-bd8f-9627831381fa None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:21.357355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c4f65-a7ba-4841-bd8f-9627831381fa None admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:21.358835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c4f65-a7ba-4841-bd8f-9627831381fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:21.359348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c4f65-a7ba-4841-bd8f-9627831381fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:21.361986 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c4f65-a7ba-4841-bd8f-9627831381fa None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:21.379255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:21.391742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 637/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Aug 1 11:54:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:21.400809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb958c5-4121-42c3-9734-444ccc0de190 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:21.540386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb958c5-4121-42c3-9734-444ccc0de190 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:21.582138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90dbee5-5294-40e0-bf61-75d45af87348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:21.582887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dbee5-5294-40e0-bf61-75d45af87348 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:21.583216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dbee5-5294-40e0-bf61-75d45af87348 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:21.583541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90dbee5-5294-40e0-bf61-75d45af87348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:21.597027 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90dbee5-5294-40e0-bf61-75d45af87348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:21.598781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90dbee5-5294-40e0-bf61-75d45af87348 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-f90dbee5-5294-40e0-bf61-75d45af87348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:54:21.616261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:21.618017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:21.619107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:54:21.619107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:54:21.619107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:21.619107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:21.619107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 632/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:21.627547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c4f65-a7ba-4841-bd8f-9627831381fa None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:21.749156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c4f65-a7ba-4841-bd8f-9627831381fa None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:21.789345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d738c8-8090-4214-bc35-5f267e0bdd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:21.790200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d738c8-8090-4214-bc35-5f267e0bdd5d None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:21.790587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d738c8-8090-4214-bc35-5f267e0bdd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:21.790953 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d738c8-8090-4214-bc35-5f267e0bdd5d None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:21.791810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d738c8-8090-4214-bc35-5f267e0bdd5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:21.792212 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d738c8-8090-4214-bc35-5f267e0bdd5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:21.793758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d738c8-8090-4214-bc35-5f267e0bdd5d None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:21.807381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 638/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:21.813615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90dbee5-5294-40e0-bf61-75d45af87348 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:21.868218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:22.001169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90dbee5-5294-40e0-bf61-75d45af87348 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:22.067215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d2fb5-50ac-4316-914a-c15fb232d6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:22.068340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d2fb5-50ac-4316-914a-c15fb232d6cf None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:22.068809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d2fb5-50ac-4316-914a-c15fb232d6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:22.069257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d2fb5-50ac-4316-914a-c15fb232d6cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:22.081839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d2fb5-50ac-4316-914a-c15fb232d6cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:22.084330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d2fb5-50ac-4316-914a-c15fb232d6cf None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-5f8d2fb5-50ac-4316-914a-c15fb232d6cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:54:22.106917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:22.108524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:22.110113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:54:22.110113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:54:22.110113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:54:22.110113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:54:22.110113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:54:22.110113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:54:22.110113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:54:22.110113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:22.110113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 633/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:22.119127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d738c8-8090-4214-bc35-5f267e0bdd5d None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:22.215136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:22.279696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d738c8-8090-4214-bc35-5f267e0bdd5d None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:22.324516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d282d4-ee4a-4fbb-a8bd-2c76994c3d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:22.325198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d282d4-ee4a-4fbb-a8bd-2c76994c3d8c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:22.325471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d282d4-ee4a-4fbb-a8bd-2c76994c3d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:22.325759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d282d4-ee4a-4fbb-a8bd-2c76994c3d8c None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:22.326590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d282d4-ee4a-4fbb-a8bd-2c76994c3d8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:22.326892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d282d4-ee4a-4fbb-a8bd-2c76994c3d8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:22.328465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d282d4-ee4a-4fbb-a8bd-2c76994c3d8c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:22.343628 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 639/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:22.355219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d2fb5-50ac-4316-914a-c15fb232d6cf None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:22.521802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:22.534209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d2fb5-50ac-4316-914a-c15fb232d6cf None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:22.580944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc249c-ddb2-4ea5-a817-d65a4f460102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:22.581712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc249c-ddb2-4ea5-a817-d65a4f460102 None admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:22.582034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc249c-ddb2-4ea5-a817-d65a4f460102 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:22.582346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc249c-ddb2-4ea5-a817-d65a4f460102 None admin] PATH_INFO: `/v3/projects/1ba6050774f047e9816a0466e688a778/users/9db663d35f964d42938be0d1a9ed4071/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:22.584912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc249c-ddb2-4ea5-a817-d65a4f460102 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba6050774f047e9816a0466e688a778, user_id=9db663d35f964d42938be0d1a9ed4071, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:22.586618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc249c-ddb2-4ea5-a817-d65a4f460102 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:22.605917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 634/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Aug 1 11:54:22 2022] PUT /identity/v3/projects/1ba6050774f047e9816a0466e688a778/users/9db663d35f964d42938be0d1a9ed4071/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 11:54:24.762291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb536d-8b09-4c00-90d4-6702e0ba0664 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:24.762927 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb536d-8b09-4c00-90d4-6702e0ba0664 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:24.762927 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb536d-8b09-4c00-90d4-6702e0ba0664 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:24.764507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 640/1274] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:54:24.771348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b016c2f-c71a-4332-9945-bddb9029350d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:24.771973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b016c2f-c71a-4332-9945-bddb9029350d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:24.771973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b016c2f-c71a-4332-9945-bddb9029350d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:24.792934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-5b016c2f-c71a-4332-9945-bddb9029350d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:54:24.794250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5b016c2f-c71a-4332-9945-bddb9029350d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:54:24.876811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 635/1275] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:54:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 01 11:54:24.900746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb536d-8b09-4c00-90d4-6702e0ba0664 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:25.028649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb536d-8b09-4c00-90d4-6702e0ba0664 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:25.049777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:25.069073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a904c-7139-4508-8542-d84369bca42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:25.069835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a904c-7139-4508-8542-d84369bca42e None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:25.070126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a904c-7139-4508-8542-d84369bca42e None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:25.070377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a904c-7139-4508-8542-d84369bca42e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:25.082921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4a904c-7139-4508-8542-d84369bca42e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-9d4a904c-7139-4508-8542-d84369bca42e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 01 11:54:25.088184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:25.090126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:25.091411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 641/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Aug 1 11:54:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:25.096127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b016c2f-c71a-4332-9945-bddb9029350d None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:25.193564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:25.216461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b016c2f-c71a-4332-9945-bddb9029350d None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:25.259910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79275b44-8af2-43e3-93ed-c16757adafe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:25.260583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79275b44-8af2-43e3-93ed-c16757adafe3 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:25.260864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79275b44-8af2-43e3-93ed-c16757adafe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:25.261134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79275b44-8af2-43e3-93ed-c16757adafe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:25.261924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79275b44-8af2-43e3-93ed-c16757adafe3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:25.262253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79275b44-8af2-43e3-93ed-c16757adafe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:25.263713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79275b44-8af2-43e3-93ed-c16757adafe3 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:25.273861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 636/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Aug 1 11:54:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:25.280726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a904c-7139-4508-8542-d84369bca42e None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:25.438572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a904c-7139-4508-8542-d84369bca42e None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:25.495494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153c723-2ab2-4b79-a51e-24b9bc1f2838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:25.496398 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153c723-2ab2-4b79-a51e-24b9bc1f2838 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:25.496841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153c723-2ab2-4b79-a51e-24b9bc1f2838 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:25.497284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153c723-2ab2-4b79-a51e-24b9bc1f2838 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:25.515352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153c723-2ab2-4b79-a51e-24b9bc1f2838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:25.518039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153c723-2ab2-4b79-a51e-24b9bc1f2838 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-c153c723-2ab2-4b79-a51e-24b9bc1f2838 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:54:25.540788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:25.542667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:25.544365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:54:25.544365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:54:25.544365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:25.544365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:25.544365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 642/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 01 11:54:25.549393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79275b44-8af2-43e3-93ed-c16757adafe3 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:25.674022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79275b44-8af2-43e3-93ed-c16757adafe3 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:25.714818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5d771-a26c-41d9-acf7-b5abc59d0bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:25.715527 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5d771-a26c-41d9-acf7-b5abc59d0bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:25.715850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5d771-a26c-41d9-acf7-b5abc59d0bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:25.716178 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5d771-a26c-41d9-acf7-b5abc59d0bf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:25.717011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5d771-a26c-41d9-acf7-b5abc59d0bf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:25.717359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5d771-a26c-41d9-acf7-b5abc59d0bf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:25.718830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5d771-a26c-41d9-acf7-b5abc59d0bf1 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:25.733204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 637/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:25.739466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153c723-2ab2-4b79-a51e-24b9bc1f2838 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:25.778169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:25.869039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153c723-2ab2-4b79-a51e-24b9bc1f2838 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:25.913156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d13ac2-79ff-4bb8-9065-63891141c329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:25.913886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d13ac2-79ff-4bb8-9065-63891141c329 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:25.914218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d13ac2-79ff-4bb8-9065-63891141c329 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:25.914540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d13ac2-79ff-4bb8-9065-63891141c329 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:25.924443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d13ac2-79ff-4bb8-9065-63891141c329 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:25.926177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d13ac2-79ff-4bb8-9065-63891141c329 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-d5d13ac2-79ff-4bb8-9065-63891141c329 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 01 11:54:25.941208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:25.943143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:25.945471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 643/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:25.950812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5d771-a26c-41d9-acf7-b5abc59d0bf1 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:25.955542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:26.080763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5d771-a26c-41d9-acf7-b5abc59d0bf1 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:26.122120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bd04a-6be1-4e04-8604-ed561fdca331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:26.122731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bd04a-6be1-4e04-8604-ed561fdca331 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:26.123059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bd04a-6be1-4e04-8604-ed561fdca331 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:26.123413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bd04a-6be1-4e04-8604-ed561fdca331 None admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:26.124431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bd04a-6be1-4e04-8604-ed561fdca331 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:26.124800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bd04a-6be1-4e04-8604-ed561fdca331 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:26.126442 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bd04a-6be1-4e04-8604-ed561fdca331 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:26.147783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 638/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Aug 1 11:54:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:26.154736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d13ac2-79ff-4bb8-9065-63891141c329 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:26.300718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:26.316326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d13ac2-79ff-4bb8-9065-63891141c329 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:26.375811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d03973-da21-465b-8fa0-e34d1eb8c59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:26.376770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d03973-da21-465b-8fa0-e34d1eb8c59f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:26.377250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d03973-da21-465b-8fa0-e34d1eb8c59f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:26.377707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d03973-da21-465b-8fa0-e34d1eb8c59f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:26.502916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d03973-da21-465b-8fa0-e34d1eb8c59f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:26.505520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d03973-da21-465b-8fa0-e34d1eb8c59f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:26.633651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:26.652369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 644/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Aug 1 11:54:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 01 11:54:26.662056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bd04a-6be1-4e04-8604-ed561fdca331 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:26.752328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:26.799392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bd04a-6be1-4e04-8604-ed561fdca331 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:26.849576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37282d9c-e3ad-4516-b4ae-28c98a56a99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:26.851166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37282d9c-e3ad-4516-b4ae-28c98a56a99b None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:26.851342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37282d9c-e3ad-4516-b4ae-28c98a56a99b None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:26.851690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37282d9c-e3ad-4516-b4ae-28c98a56a99b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:26.864317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37282d9c-e3ad-4516-b4ae-28c98a56a99b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:26.865868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37282d9c-e3ad-4516-b4ae-28c98a56a99b None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-37282d9c-e3ad-4516-b4ae-28c98a56a99b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 01 11:54:26.883374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:26.884709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:26.885931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 01 11:54:26.885931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 01 11:54:26.885931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 01 11:54:26.885931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:26.885931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 639/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:26.891679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d03973-da21-465b-8fa0-e34d1eb8c59f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:27.018395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d03973-da21-465b-8fa0-e34d1eb8c59f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:27.061722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaa96c-f633-4315-b7e0-cd5f74fd5848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:27.062361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaa96c-f633-4315-b7e0-cd5f74fd5848 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:27.062651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaa96c-f633-4315-b7e0-cd5f74fd5848 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:27.062917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaa96c-f633-4315-b7e0-cd5f74fd5848 None admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:27.063673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaa96c-f633-4315-b7e0-cd5f74fd5848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:27.063977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaa96c-f633-4315-b7e0-cd5f74fd5848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:27.065574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaa96c-f633-4315-b7e0-cd5f74fd5848 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:27.079876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 645/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Aug 1 11:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:27.086765 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37282d9c-e3ad-4516-b4ae-28c98a56a99b None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:27.212924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37282d9c-e3ad-4516-b4ae-28c98a56a99b None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:27.229573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:27.254203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eaf01-e211-4465-8aa0-a5e43e89f2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:27.254893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eaf01-e211-4465-8aa0-a5e43e89f2eb None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:27.255172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eaf01-e211-4465-8aa0-a5e43e89f2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:27.255440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eaf01-e211-4465-8aa0-a5e43e89f2eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:27.263791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4eaf01-e211-4465-8aa0-a5e43e89f2eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:27.265623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4eaf01-e211-4465-8aa0-a5e43e89f2eb None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application [None req-8f4eaf01-e211-4465-8aa0-a5e43e89f2eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 01 11:54:27.279956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:27.281689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:27.283323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 01 11:54:27.283323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 01 11:54:27.283323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 01 11:54:27.283323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 01 11:54:27.283323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 01 11:54:27.283323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 01 11:54:27.283323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 01 11:54:27.283323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: ERROR keystone.server.flask.application Aug 01 11:54:27.283323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 640/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Aug 1 11:54:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:27.291068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaa96c-f633-4315-b7e0-cd5f74fd5848 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:27.350275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:27.416335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaa96c-f633-4315-b7e0-cd5f74fd5848 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:27.456775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0141240-a545-48fc-83cd-9f6d4c1fb871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:27.457477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0141240-a545-48fc-83cd-9f6d4c1fb871 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:27.457762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0141240-a545-48fc-83cd-9f6d4c1fb871 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:27.458035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0141240-a545-48fc-83cd-9f6d4c1fb871 None admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:27.458878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0141240-a545-48fc-83cd-9f6d4c1fb871 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:27.459209 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0141240-a545-48fc-83cd-9f6d4c1fb871 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:27.460812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0141240-a545-48fc-83cd-9f6d4c1fb871 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:27.475705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 646/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Aug 1 11:54:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:27.483074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eaf01-e211-4465-8aa0-a5e43e89f2eb None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:27.603567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eaf01-e211-4465-8aa0-a5e43e89f2eb None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:27.655755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dec58-c05f-4a45-8ec9-5962884ee1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:27.657015 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dec58-c05f-4a45-8ec9-5962884ee1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:27.657537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dec58-c05f-4a45-8ec9-5962884ee1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:27.658034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dec58-c05f-4a45-8ec9-5962884ee1f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:27.659651 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dec58-c05f-4a45-8ec9-5962884ee1f0 None admin] RBAC: Adding query filter params (role.id=696d4e7be8f44c6da474e2a3ac0133c3, scope.project.id=1ba6050774f047e9816a0466e688a778, user.id=9db663d35f964d42938be0d1a9ed4071) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:27.660202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dec58-c05f-4a45-8ec9-5962884ee1f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:27.663191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dec58-c05f-4a45-8ec9-5962884ee1f0 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:27.676115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 641/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Aug 1 11:54:27 2022] GET /identity/v3/role_assignments?user.id=9db663d35f964d42938be0d1a9ed4071&scope.project.id=1ba6050774f047e9816a0466e688a778&role.id=696d4e7be8f44c6da474e2a3ac0133c3 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:54:29.805846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9919dda-4da0-4ab8-ac83-bc523e193b63 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:29.806562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9919dda-4da0-4ab8-ac83-bc523e193b63 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:29.806787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9919dda-4da0-4ab8-ac83-bc523e193b63 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:29.809149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 647/1288] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:54:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:54:29.817622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cdf7c-1846-4e0e-a5cb-b77857fe7f43 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:29.818208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cdf7c-1846-4e0e-a5cb-b77857fe7f43 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:29.818394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cdf7c-1846-4e0e-a5cb-b77857fe7f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:29.840963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-509cdf7c-1846-4e0e-a5cb-b77857fe7f43 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:54:29.842270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-509cdf7c-1846-4e0e-a5cb-b77857fe7f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:54:29.881357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:29.929425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 642/1289] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:54:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:54:29.955198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9919dda-4da0-4ab8-ac83-bc523e193b63 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:30.155544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9919dda-4da0-4ab8-ac83-bc523e193b63 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:30.178867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:30.227830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a93364-349d-4ce1-9987-bffe2f657472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:30.228983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a93364-349d-4ce1-9987-bffe2f657472 None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:30.229477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a93364-349d-4ce1-9987-bffe2f657472 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:30.229962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a93364-349d-4ce1-9987-bffe2f657472 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:30.231147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a93364-349d-4ce1-9987-bffe2f657472 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:30.233838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a93364-349d-4ce1-9987-bffe2f657472 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-11a93364-349d-4ce1-9987-bffe2f657472 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:30.252179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:30.253906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:30.255886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 648/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Aug 1 11:54:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:30.262518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cdf7c-1846-4e0e-a5cb-b77857fe7f43 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:30.384048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cdf7c-1846-4e0e-a5cb-b77857fe7f43 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:30.429240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15d757-3ba7-4937-8313-862c6dde3894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:30.430600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d757-3ba7-4937-8313-862c6dde3894 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:30.431294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d757-3ba7-4937-8313-862c6dde3894 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:30.432071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d757-3ba7-4937-8313-862c6dde3894 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:30.433957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15d757-3ba7-4937-8313-862c6dde3894 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:30.434568 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15d757-3ba7-4937-8313-862c6dde3894 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:30.436730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15d757-3ba7-4937-8313-862c6dde3894 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:30.442499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 643/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Aug 1 11:54:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:30.448868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a93364-349d-4ce1-9987-bffe2f657472 None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:30.558855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a93364-349d-4ce1-9987-bffe2f657472 None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:30.598098 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fed05f-00bb-42dd-8c18-c3f757e7948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:30.598803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fed05f-00bb-42dd-8c18-c3f757e7948b None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:30.599134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fed05f-00bb-42dd-8c18-c3f757e7948b None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:30.599467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fed05f-00bb-42dd-8c18-c3f757e7948b None admin] PATH_INFO: `/v3/services` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:30.600621 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fed05f-00bb-42dd-8c18-c3f757e7948b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:30.600970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fed05f-00bb-42dd-8c18-c3f757e7948b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:30.602496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fed05f-00bb-42dd-8c18-c3f757e7948b None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:30.607972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 649/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Aug 1 11:54:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:33.103841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa58805-452b-4021-a158-90898c05203a None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:33.104508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa58805-452b-4021-a158-90898c05203a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:33.104508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa58805-452b-4021-a158-90898c05203a None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:33.105668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 644/1293] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:54:33.111914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1bd92-4825-4433-8c0b-aa74aef9e3e5 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:33.112512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1bd92-4825-4433-8c0b-aa74aef9e3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:33.112742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1bd92-4825-4433-8c0b-aa74aef9e3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:33.136042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8eb1bd92-4825-4433-8c0b-aa74aef9e3e5 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:54:33.137314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8eb1bd92-4825-4433-8c0b-aa74aef9e3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:54:33.172649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:33.218084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 650/1294] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:54:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:54:33.243708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa58805-452b-4021-a158-90898c05203a None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:33.282795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:33.385685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa58805-452b-4021-a158-90898c05203a None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:33.435791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f59405-872e-4aec-99d4-3d406a960237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:33.436578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f59405-872e-4aec-99d4-3d406a960237 None admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:33.436992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f59405-872e-4aec-99d4-3d406a960237 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:33.437331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f59405-872e-4aec-99d4-3d406a960237 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:33.438494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f59405-872e-4aec-99d4-3d406a960237 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:33.440762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f59405-872e-4aec-99d4-3d406a960237 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:33.464352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 645/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Aug 1 11:54:33 2022] POST /identity/v3/services => generated 246 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:54:35.537622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f8c79-a92e-43f9-8de3-1cf944f3d8d8 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:35.538256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f8c79-a92e-43f9-8de3-1cf944f3d8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:35.538256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f8c79-a92e-43f9-8de3-1cf944f3d8d8 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:35.539566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 651/1296] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:54:35.545767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32fce4-3e15-479f-8aa4-f0e09c827592 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:35.546345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32fce4-3e15-479f-8aa4-f0e09c827592 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:35.546345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32fce4-3e15-479f-8aa4-f0e09c827592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:35.566555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-da32fce4-3e15-479f-8aa4-f0e09c827592 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:54:35.567849 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-da32fce4-3e15-479f-8aa4-f0e09c827592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:54:35.654150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 646/1297] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:54:35 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:54:35.677776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f8c79-a92e-43f9-8de3-1cf944f3d8d8 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:35.802001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f8c79-a92e-43f9-8de3-1cf944f3d8d8 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:35.843530 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292b0ab-c9df-452f-8c6b-590fdb43648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:35.844553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292b0ab-c9df-452f-8c6b-590fdb43648f None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:35.844893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292b0ab-c9df-452f-8c6b-590fdb43648f None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:35.845201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292b0ab-c9df-452f-8c6b-590fdb43648f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:35.845897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292b0ab-c9df-452f-8c6b-590fdb43648f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:35.847385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292b0ab-c9df-452f-8c6b-590fdb43648f None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-6292b0ab-c9df-452f-8c6b-590fdb43648f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:35.859341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:35.861091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:35.862365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 652/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Aug 1 11:54:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:35.869246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32fce4-3e15-479f-8aa4-f0e09c827592 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:35.892444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:35.991956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32fce4-3e15-479f-8aa4-f0e09c827592 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:36.031040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e15d-b2f6-4734-9893-9f122ad3a5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:36.031684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e15d-b2f6-4734-9893-9f122ad3a5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:36.031959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e15d-b2f6-4734-9893-9f122ad3a5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:36.032227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e15d-b2f6-4734-9893-9f122ad3a5a3 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:36.032958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854e15d-b2f6-4734-9893-9f122ad3a5a3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:36.033263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854e15d-b2f6-4734-9893-9f122ad3a5a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:36.034674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854e15d-b2f6-4734-9893-9f122ad3a5a3 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:36.040751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 647/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Aug 1 11:54:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 11:54:36.047646 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292b0ab-c9df-452f-8c6b-590fdb43648f None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:36.085042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:36.166228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292b0ab-c9df-452f-8c6b-590fdb43648f None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:36.204708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9e6d8-8370-4705-aad5-dcb8feb4892c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:36.205343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9e6d8-8370-4705-aad5-dcb8feb4892c None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:36.205620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9e6d8-8370-4705-aad5-dcb8feb4892c None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:36.205893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9e6d8-8370-4705-aad5-dcb8feb4892c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:36.206674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9e6d8-8370-4705-aad5-dcb8feb4892c None admin] RBAC: Adding query filter params (interface=public, service_id=4a3a5cfd753b460d8bc26fa81c318e63, region_id=RegionOne) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:36.206985 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9e6d8-8370-4705-aad5-dcb8feb4892c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:36.208426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9e6d8-8370-4705-aad5-dcb8feb4892c None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:36.214718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 653/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Aug 1 11:54:36 2022] GET /identity/v3/endpoints?service_id=4a3a5cfd753b460d8bc26fa81c318e63&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:54:36.221130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e15d-b2f6-4734-9893-9f122ad3a5a3 None admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:36.316183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:36.337202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e15d-b2f6-4734-9893-9f122ad3a5a3 None admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:36.379587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79632bbf-41aa-4e36-8120-cb55d6188888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:36.380346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79632bbf-41aa-4e36-8120-cb55d6188888 None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:36.380676 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79632bbf-41aa-4e36-8120-cb55d6188888 None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:36.380987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79632bbf-41aa-4e36-8120-cb55d6188888 None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:36.381723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79632bbf-41aa-4e36-8120-cb55d6188888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:36.386655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79632bbf-41aa-4e36-8120-cb55d6188888 None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:36.396864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 648/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Aug 1 11:54:36 2022] GET /identity/v3/services => generated 2191 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 11:54:38.515991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eac74b-41ea-4234-85a9-f988372e571e None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:38.516688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eac74b-41ea-4234-85a9-f988372e571e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:38.516688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eac74b-41ea-4234-85a9-f988372e571e None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:38.517933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 654/1302] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:54:38.524558 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617139-7432-446b-ab87-64e6de00c55d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:38.525137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617139-7432-446b-ab87-64e6de00c55d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:38.525137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617139-7432-446b-ab87-64e6de00c55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:38.545142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-61617139-7432-446b-ab87-64e6de00c55d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:54:38.546419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-61617139-7432-446b-ab87-64e6de00c55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:54:38.626089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 649/1303] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:54:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 01 11:54:38.649838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eac74b-41ea-4234-85a9-f988372e571e None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:38.831861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:38.844836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eac74b-41ea-4234-85a9-f988372e571e None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:38.890774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0d1d8-3ba5-4c82-a0ad-5a29fb05dcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:38.891509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0d1d8-3ba5-4c82-a0ad-5a29fb05dcef None admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:38.891835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0d1d8-3ba5-4c82-a0ad-5a29fb05dcef None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:38.892161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0d1d8-3ba5-4c82-a0ad-5a29fb05dcef None admin] PATH_INFO: `/v3/services/placement` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:38.892980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0d1d8-3ba5-4c82-a0ad-5a29fb05dcef None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:38.894535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0d1d8-3ba5-4c82-a0ad-5a29fb05dcef None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application [None req-80f0d1d8-3ba5-4c82-a0ad-5a29fb05dcef None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 01 11:54:38.905051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 01 11:54:38.907354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.get_or_create( Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application with Lock( Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self._enter() Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application return self.creator() Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application created_value = creator( Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: ERROR keystone.server.flask.application Aug 01 11:54:38.909339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 655/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Aug 1 11:54:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 01 11:54:38.913429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61617139-7432-446b-ab87-64e6de00c55d None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:39.003602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:39.032780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61617139-7432-446b-ab87-64e6de00c55d None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:39.074010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674eb273-e187-485f-879a-8602a3353c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:39.074726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674eb273-e187-485f-879a-8602a3353c3d None admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:39.075011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674eb273-e187-485f-879a-8602a3353c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:39.075288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674eb273-e187-485f-879a-8602a3353c3d None admin] PATH_INFO: `/v3/services` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:39.075996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674eb273-e187-485f-879a-8602a3353c3d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 11:54:39.076331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674eb273-e187-485f-879a-8602a3353c3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:39.077769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674eb273-e187-485f-879a-8602a3353c3d None admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:39.084198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 650/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Aug 1 11:54:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 11:54:39.093765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0d1d8-3ba5-4c82-a0ad-5a29fb05dcef None admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:54:39.213634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0d1d8-3ba5-4c82-a0ad-5a29fb05dcef None admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:54:39.255140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e4533-daf5-4d57-9be1-4f34f29693f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:54:39.255782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e4533-daf5-4d57-9be1-4f34f29693f0 None admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:54:39.256065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e4533-daf5-4d57-9be1-4f34f29693f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:54:39.256352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e4533-daf5-4d57-9be1-4f34f29693f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:54:39.257100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e4533-daf5-4d57-9be1-4f34f29693f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:54:39.258602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e4533-daf5-4d57-9be1-4f34f29693f0 None admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:54:39.282084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 656/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 11:54:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 11:54:49.081447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=30 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:54:49.277625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:55:51.973441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe395d-44f1-49c3-aec0-eb6b30909e81 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:55:51.974341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe395d-44f1-49c3-aec0-eb6b30909e81 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:55:51.974341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe395d-44f1-49c3-aec0-eb6b30909e81 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:55:51.977041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 651/1307] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:55:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:55:51.985478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132476f-8050-48a7-aa07-c9b5483915b5 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:55:51.986384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132476f-8050-48a7-aa07-c9b5483915b5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:55:51.986384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132476f-8050-48a7-aa07-c9b5483915b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:55:52.134507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8132476f-8050-48a7-aa07-c9b5483915b5 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:55:52.136651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8132476f-8050-48a7-aa07-c9b5483915b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:55:52.355076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 657/1308] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:55:52.440345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d43211-deb8-4eb1-bbee-63c1b827d5bc None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:55:52.441117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d43211-deb8-4eb1-bbee-63c1b827d5bc None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:55:52.441276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d43211-deb8-4eb1-bbee-63c1b827d5bc None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:55:52.445095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 652/1309] 10.222.0.49 () {54 vars in 965 bytes} [Mon Aug 1 11:55:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:55:52.456167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be5fdf-0334-4b0f-ad43-119a78b5c148 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:55:52.457002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be5fdf-0334-4b0f-ad43-119a78b5c148 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:55:52.457882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be5fdf-0334-4b0f-ad43-119a78b5c148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:55:52.611616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-78be5fdf-0334-4b0f-ad43-119a78b5c148 None None] MFA Rules not processed for user `8dfbda2838b242558c885f913fafb5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:55:52.613572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-78be5fdf-0334-4b0f-ad43-119a78b5c148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:55:52.662639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:55:52.791363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 658/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Mon Aug 1 11:55:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:55:52.808984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d43211-deb8-4eb1-bbee-63c1b827d5bc None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:55:52.812714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d1d43211-deb8-4eb1-bbee-63c1b827d5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:55:53.076463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d43211-deb8-4eb1-bbee-63c1b827d5bc None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:55:53.243004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc04e8-80af-41c0-a5b4-27be2e1737c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:55:53.244466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc04e8-80af-41c0-a5b4-27be2e1737c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:55:53.245161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc04e8-80af-41c0-a5b4-27be2e1737c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:55:53.245637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc04e8-80af-41c0-a5b4-27be2e1737c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:55:53.249475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-68fc04e8-80af-41c0-a5b4-27be2e1737c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:55:53.356417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:55:53.441721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc04e8-80af-41c0-a5b4-27be2e1737c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:55:53.443502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc04e8-80af-41c0-a5b4-27be2e1737c4 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:55:53.472556 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 653/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:55:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:55:56.281045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667c54b-015a-4e7e-bbb8-6ac532ef8603 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:55:56.281665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667c54b-015a-4e7e-bbb8-6ac532ef8603 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:55:56.281996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667c54b-015a-4e7e-bbb8-6ac532ef8603 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:55:56.285829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 659/1312] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:55:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:55:56.291685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352758-0631-4862-8c81-1d6bce1c9818 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:55:56.292483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352758-0631-4862-8c81-1d6bce1c9818 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:55:56.292483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07352758-0631-4862-8c81-1d6bce1c9818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:55:56.396413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-07352758-0631-4862-8c81-1d6bce1c9818 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:55:56.397988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-07352758-0631-4862-8c81-1d6bce1c9818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:55:56.522908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 654/1313] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:55:56.585664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667c54b-015a-4e7e-bbb8-6ac532ef8603 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:55:56.587431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f667c54b-015a-4e7e-bbb8-6ac532ef8603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:55:56.721207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667c54b-015a-4e7e-bbb8-6ac532ef8603 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:55:56.818538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:55:56.821499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40637a-8f55-4bfb-85e1-cbf2457766ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:55:56.822226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40637a-8f55-4bfb-85e1-cbf2457766ce service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:55:56.822784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40637a-8f55-4bfb-85e1-cbf2457766ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:55:56.823323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40637a-8f55-4bfb-85e1-cbf2457766ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:55:56.825486 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7e40637a-8f55-4bfb-85e1-cbf2457766ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:55:56.973685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40637a-8f55-4bfb-85e1-cbf2457766ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:55:56.975296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40637a-8f55-4bfb-85e1-cbf2457766ce service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:55:56.999653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 660/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:55:59.512891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93da34-ded5-451d-9d9b-a3fe242b598f None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:55:59.513784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93da34-ded5-451d-9d9b-a3fe242b598f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:55:59.513784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93da34-ded5-451d-9d9b-a3fe242b598f None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:55:59.515402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 655/1315] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:55:59.521840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140cd5c-a19d-4562-b532-50a74946caf0 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:55:59.522743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140cd5c-a19d-4562-b532-50a74946caf0 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:55:59.522743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140cd5c-a19d-4562-b532-50a74946caf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:55:59.604221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-1140cd5c-a19d-4562-b532-50a74946caf0 None None] MFA Rules not processed for user `32192a9fb66c412e97d89044a31c5688`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:55:59.605643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1140cd5c-a19d-4562-b532-50a74946caf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:55:59.695304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:55:59.778933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 661/1316] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:55:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:55:59.838947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93da34-ded5-451d-9d9b-a3fe242b598f None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:55:59.840546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-bf93da34-ded5-451d-9d9b-a3fe242b598f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:55:59.853621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:55:59.993233 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93da34-ded5-451d-9d9b-a3fe242b598f None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:00.082152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc249c-33ae-4456-9b22-63e194e1da46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:00.082820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc249c-33ae-4456-9b22-63e194e1da46 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:00.083133 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc249c-33ae-4456-9b22-63e194e1da46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:00.083531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc249c-33ae-4456-9b22-63e194e1da46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:00.085323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-87dc249c-33ae-4456-9b22-63e194e1da46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:00.163113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:00.219143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc249c-33ae-4456-9b22-63e194e1da46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:00.221288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc249c-33ae-4456-9b22-63e194e1da46 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:00.243668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 656/1317] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:55:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:56:03.060275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1cbab-eea8-4bd7-8c84-6973614fcdd0 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:03.060879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1cbab-eea8-4bd7-8c84-6973614fcdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:03.061047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1cbab-eea8-4bd7-8c84-6973614fcdd0 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:03.062611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 662/1318] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:03.068981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2a870-d6cd-4b51-97f0-78bb2ada522d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:03.069584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2a870-d6cd-4b51-97f0-78bb2ada522d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:03.069584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2a870-d6cd-4b51-97f0-78bb2ada522d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:03.158279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-6ec2a870-d6cd-4b51-97f0-78bb2ada522d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:03.159671 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6ec2a870-d6cd-4b51-97f0-78bb2ada522d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:03.275842 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 657/1319] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:56:03.342630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1cbab-eea8-4bd7-8c84-6973614fcdd0 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:03.344238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-edc1cbab-eea8-4bd7-8c84-6973614fcdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:03.475587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1cbab-eea8-4bd7-8c84-6973614fcdd0 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:03.567469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22da41a-4482-4202-8923-ed2ba6b485b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:03.568127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22da41a-4482-4202-8923-ed2ba6b485b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:03.568443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22da41a-4482-4202-8923-ed2ba6b485b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:03.568716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22da41a-4482-4202-8923-ed2ba6b485b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:03.570466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c22da41a-4482-4202-8923-ed2ba6b485b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:03.579637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:03.707192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22da41a-4482-4202-8923-ed2ba6b485b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:03.708810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22da41a-4482-4202-8923-ed2ba6b485b6 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:03.732010 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 663/1320] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:56:07.141518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40501c22-124d-4e72-8603-e6291fc5c6c1 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:07.142483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40501c22-124d-4e72-8603-e6291fc5c6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:07.142483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40501c22-124d-4e72-8603-e6291fc5c6c1 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:07.144714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 658/1321] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:07.154034 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c52ad-014b-46bd-8fd4-f2a1652375af None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:07.154642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c52ad-014b-46bd-8fd4-f2a1652375af None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:07.154875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c52ad-014b-46bd-8fd4-f2a1652375af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:07.274085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-153c52ad-014b-46bd-8fd4-f2a1652375af None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:07.275489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-153c52ad-014b-46bd-8fd4-f2a1652375af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:07.352156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:07.399923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 664/1322] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:07.442462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40501c22-124d-4e72-8603-e6291fc5c6c1 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:07.445330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-40501c22-124d-4e72-8603-e6291fc5c6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:07.471610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:07.618771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40501c22-124d-4e72-8603-e6291fc5c6c1 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:07.756132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223f943-473c-4cea-a045-6d3fa661aea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:07.757063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223f943-473c-4cea-a045-6d3fa661aea7 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:07.757347 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223f943-473c-4cea-a045-6d3fa661aea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:07.757643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223f943-473c-4cea-a045-6d3fa661aea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:07.759625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7223f943-473c-4cea-a045-6d3fa661aea7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:07.886410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:07.943700 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223f943-473c-4cea-a045-6d3fa661aea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:07.945527 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223f943-473c-4cea-a045-6d3fa661aea7 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:07.974078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 659/1323] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:56:10.203451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b651-6860-42d5-a76f-09fac788be2c None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:10.204097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b651-6860-42d5-a76f-09fac788be2c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:10.204274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc6b651-6860-42d5-a76f-09fac788be2c None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:10.207708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 665/1324] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:10.219317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fab57-a1eb-4d3b-a640-e5bd3e0fc19c None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:10.220170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fab57-a1eb-4d3b-a640-e5bd3e0fc19c None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:10.220745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fab57-a1eb-4d3b-a640-e5bd3e0fc19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:10.319015 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-017fab57-a1eb-4d3b-a640-e5bd3e0fc19c None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:10.320370 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-017fab57-a1eb-4d3b-a640-e5bd3e0fc19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:10.453864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 660/1325] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:10.507183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eca171-c888-40b0-a8ea-4959b4a302e2 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:10.507183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eca171-c888-40b0-a8ea-4959b4a302e2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:10.507554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eca171-c888-40b0-a8ea-4959b4a302e2 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:10.509670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 666/1326] 10.222.0.49 () {54 vars in 965 bytes} [Mon Aug 1 11:56:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:10.517384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e282f7-a14d-41a4-ba5c-82aab8351f51 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:10.517732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e282f7-a14d-41a4-ba5c-82aab8351f51 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:10.518139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e282f7-a14d-41a4-ba5c-82aab8351f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:10.559752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:10.615579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e5e282f7-a14d-41a4-ba5c-82aab8351f51 None None] MFA Rules not processed for user `8dfbda2838b242558c885f913fafb5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:10.616901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e5e282f7-a14d-41a4-ba5c-82aab8351f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:10.712672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 661/1327] 10.222.0.49 () {58 vars in 1062 bytes} [Mon Aug 1 11:56:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:10.723306 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eca171-c888-40b0-a8ea-4959b4a302e2 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:10.724918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-22eca171-c888-40b0-a8ea-4959b4a302e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:10.858600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eca171-c888-40b0-a8ea-4959b4a302e2 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:10.952697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda9509c-0c74-4e64-b0c1-eb7370755060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:10.953346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9509c-0c74-4e64-b0c1-eb7370755060 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:10.953620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9509c-0c74-4e64-b0c1-eb7370755060 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:10.953989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda9509c-0c74-4e64-b0c1-eb7370755060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:10.955746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-dda9509c-0c74-4e64-b0c1-eb7370755060 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:10.976000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:11.093448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9509c-0c74-4e64-b0c1-eb7370755060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:11.095076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda9509c-0c74-4e64-b0c1-eb7370755060 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:11.118445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 667/1328] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:56:13.520086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05765c-99d7-4f9a-984d-2729edff89bd None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:13.520903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05765c-99d7-4f9a-984d-2729edff89bd None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:13.520903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05765c-99d7-4f9a-984d-2729edff89bd None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:13.522557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 662/1329] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:13.529888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78252d09-6a5b-4ed5-aca7-db3bd5aa9da0 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:13.530654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78252d09-6a5b-4ed5-aca7-db3bd5aa9da0 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:13.530793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78252d09-6a5b-4ed5-aca7-db3bd5aa9da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:13.638778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-78252d09-6a5b-4ed5-aca7-db3bd5aa9da0 None None] MFA Rules not processed for user `32192a9fb66c412e97d89044a31c5688`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:13.640164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-78252d09-6a5b-4ed5-aca7-db3bd5aa9da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:13.724313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:13.780722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 668/1330] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:13.825065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05765c-99d7-4f9a-984d-2729edff89bd None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:13.827014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7a05765c-99d7-4f9a-984d-2729edff89bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:13.967103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05765c-99d7-4f9a-984d-2729edff89bd None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:13.995061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:14.092157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e2af7-686c-40a1-830b-e37455a55720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:14.093244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e2af7-686c-40a1-830b-e37455a55720 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:14.093749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e2af7-686c-40a1-830b-e37455a55720 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:14.094196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e2af7-686c-40a1-830b-e37455a55720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:14.097724 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d80e2af7-686c-40a1-830b-e37455a55720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:14.284833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e2af7-686c-40a1-830b-e37455a55720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:14.286317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e2af7-686c-40a1-830b-e37455a55720 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:14.307408 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 663/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:56:16.837994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e509a-19a3-4e2b-a921-1102527ab9dc None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:16.838793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e509a-19a3-4e2b-a921-1102527ab9dc None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:16.838793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e509a-19a3-4e2b-a921-1102527ab9dc None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:16.844425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 669/1332] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:16.846271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a4cfd-7c12-4989-9a44-47da61e738bb None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:16.846778 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a4cfd-7c12-4989-9a44-47da61e738bb None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:16.846778 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a4cfd-7c12-4989-9a44-47da61e738bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:16.921978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:16.925240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a41a4cfd-7c12-4989-9a44-47da61e738bb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:16.928450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a41a4cfd-7c12-4989-9a44-47da61e738bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:17.043159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 664/1333] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:56:17.083558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e509a-19a3-4e2b-a921-1102527ab9dc None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:17.085782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-df0e509a-19a3-4e2b-a921-1102527ab9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:17.251385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e509a-19a3-4e2b-a921-1102527ab9dc None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:17.362634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b9ece-69fb-4b01-a768-bd5414689923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:17.363269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b9ece-69fb-4b01-a768-bd5414689923 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:17.363545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b9ece-69fb-4b01-a768-bd5414689923 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:17.363811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b9ece-69fb-4b01-a768-bd5414689923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:17.365577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-492b9ece-69fb-4b01-a768-bd5414689923 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:17.399078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:17.516681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b9ece-69fb-4b01-a768-bd5414689923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:17.518313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b9ece-69fb-4b01-a768-bd5414689923 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:17.540536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 670/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:56:19.745702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a7c54-9082-46df-abb3-815a4456ceed None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:19.746429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a7c54-9082-46df-abb3-815a4456ceed None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:19.746429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a7c54-9082-46df-abb3-815a4456ceed None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:19.747634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 665/1335] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:19.756672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127f1d1-08db-4239-83c9-e12ff61195d0 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:19.757292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127f1d1-08db-4239-83c9-e12ff61195d0 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:19.757292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a127f1d1-08db-4239-83c9-e12ff61195d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:19.857007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a127f1d1-08db-4239-83c9-e12ff61195d0 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:19.859154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a127f1d1-08db-4239-83c9-e12ff61195d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:19.999532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:20.028986 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 671/1336] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:20.070508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a7c54-9082-46df-abb3-815a4456ceed None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:20.072136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f13a7c54-9082-46df-abb3-815a4456ceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:20.216467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a7c54-9082-46df-abb3-815a4456ceed None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:20.249146 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:20.302176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c0859-f53c-4406-99e0-1ea293e0acc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:20.302717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c0859-f53c-4406-99e0-1ea293e0acc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:20.302989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c0859-f53c-4406-99e0-1ea293e0acc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:20.303254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c0859-f53c-4406-99e0-1ea293e0acc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:20.305082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-461c0859-f53c-4406-99e0-1ea293e0acc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:20.442047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c0859-f53c-4406-99e0-1ea293e0acc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:20.443542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c0859-f53c-4406-99e0-1ea293e0acc8 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:20.467014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 666/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:56:23.192962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cba57-d0d6-447a-846b-2d782c8c2c92 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:23.193829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cba57-d0d6-447a-846b-2d782c8c2c92 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:23.193829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cba57-d0d6-447a-846b-2d782c8c2c92 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:23.194807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 672/1338] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:23.202595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af660a7-e12f-4312-88ca-7042b7838a91 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:23.203712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af660a7-e12f-4312-88ca-7042b7838a91 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:23.203712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af660a7-e12f-4312-88ca-7042b7838a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:23.320427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-8af660a7-e12f-4312-88ca-7042b7838a91 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:23.322537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8af660a7-e12f-4312-88ca-7042b7838a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:23.338859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:23.448041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 667/1339] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:23.487361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cba57-d0d6-447a-846b-2d782c8c2c92 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:23.489221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9c6cba57-d0d6-447a-846b-2d782c8c2c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:23.616979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cba57-d0d6-447a-846b-2d782c8c2c92 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:23.704948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c0cad-b820-429c-a5f0-82290d564c48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:23.705504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c0cad-b820-429c-a5f0-82290d564c48 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:23.705779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c0cad-b820-429c-a5f0-82290d564c48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:23.706048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c0cad-b820-429c-a5f0-82290d564c48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:23.707817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-509c0cad-b820-429c-a5f0-82290d564c48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:23.752432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:23.861337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c0cad-b820-429c-a5f0-82290d564c48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:23.862833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c0cad-b820-429c-a5f0-82290d564c48 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:23.884699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 673/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:56:28.816877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fd5e0-7905-42dc-95da-e5a37975a875 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:28.817499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fd5e0-7905-42dc-95da-e5a37975a875 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:28.817864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fd5e0-7905-42dc-95da-e5a37975a875 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:28.820545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 668/1341] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:28.830568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2bf5f-14c2-408c-af4a-1b05a006006d None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:28.831320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2bf5f-14c2-408c-af4a-1b05a006006d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:28.831537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf2bf5f-14c2-408c-af4a-1b05a006006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:28.961946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-fcf2bf5f-14c2-408c-af4a-1b05a006006d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:28.963765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-fcf2bf5f-14c2-408c-af4a-1b05a006006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:29.077477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:29.096475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 674/1342] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:29.136815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fd5e0-7905-42dc-95da-e5a37975a875 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:29.138575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b84fd5e0-7905-42dc-95da-e5a37975a875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:29.266592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fd5e0-7905-42dc-95da-e5a37975a875 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:29.313628 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:29.360929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89c78c-5069-4edd-9631-4fe614966bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:29.361581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89c78c-5069-4edd-9631-4fe614966bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:29.361866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89c78c-5069-4edd-9631-4fe614966bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:29.362141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89c78c-5069-4edd-9631-4fe614966bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:29.363938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1c89c78c-5069-4edd-9631-4fe614966bf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:29.506321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89c78c-5069-4edd-9631-4fe614966bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:29.509280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89c78c-5069-4edd-9631-4fe614966bf4 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:29.547315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 669/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:56:32.845968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdf9d1-fa46-425e-aeec-8ed8b68bea0d None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:32.846800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdf9d1-fa46-425e-aeec-8ed8b68bea0d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:32.846954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cdf9d1-fa46-425e-aeec-8ed8b68bea0d None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:32.850342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 675/1344] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:32.858448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c68d99-d7bd-4f9c-a1f8-80f70faf2af8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:32.859375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c68d99-d7bd-4f9c-a1f8-80f70faf2af8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:32.859375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c68d99-d7bd-4f9c-a1f8-80f70faf2af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:32.959849 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-46c68d99-d7bd-4f9c-a1f8-80f70faf2af8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:32.961353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-46c68d99-d7bd-4f9c-a1f8-80f70faf2af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:32.986217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:33.084246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 670/1345] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:33.126451 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdf9d1-fa46-425e-aeec-8ed8b68bea0d None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:33.128172 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-31cdf9d1-fa46-425e-aeec-8ed8b68bea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:33.265712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cdf9d1-fa46-425e-aeec-8ed8b68bea0d None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:33.365297 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441cd621-f818-41a1-8957-9de35a7f2c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:33.366354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cd621-f818-41a1-8957-9de35a7f2c86 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:33.366825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cd621-f818-41a1-8957-9de35a7f2c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:33.367286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441cd621-f818-41a1-8957-9de35a7f2c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:33.370662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-441cd621-f818-41a1-8957-9de35a7f2c86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:33.455991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:33.552894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cd621-f818-41a1-8957-9de35a7f2c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:33.554465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441cd621-f818-41a1-8957-9de35a7f2c86 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:33.576976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 676/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:56:38.911217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ffbc-9de8-4702-ae97-f09eacad40fe None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:38.911807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ffbc-9de8-4702-ae97-f09eacad40fe None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:38.912281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ffbc-9de8-4702-ae97-f09eacad40fe None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:38.915011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 671/1347] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:38.923533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c173a-4f9f-40a8-a3a4-7222a2ac356c None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:38.924432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c173a-4f9f-40a8-a3a4-7222a2ac356c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:38.924739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c173a-4f9f-40a8-a3a4-7222a2ac356c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:39.053039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-aa6c173a-4f9f-40a8-a3a4-7222a2ac356c None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:39.055145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-aa6c173a-4f9f-40a8-a3a4-7222a2ac356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:39.185613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:39.188815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 677/1348] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:39.230210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9ffbc-9de8-4702-ae97-f09eacad40fe None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:39.231784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-22a9ffbc-9de8-4702-ae97-f09eacad40fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:39.363976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9ffbc-9de8-4702-ae97-f09eacad40fe None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:39.446853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:39.492686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea695915-810f-4ab3-aab1-2571bb6e9df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:39.493740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea695915-810f-4ab3-aab1-2571bb6e9df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:39.494206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea695915-810f-4ab3-aab1-2571bb6e9df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:39.494670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea695915-810f-4ab3-aab1-2571bb6e9df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:39.497707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ea695915-810f-4ab3-aab1-2571bb6e9df7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:39.682337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea695915-810f-4ab3-aab1-2571bb6e9df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:39.685277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea695915-810f-4ab3-aab1-2571bb6e9df7 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:39.727245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 672/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 11:56:45.456797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5063-9931-4f89-b7ee-dc40606415e4 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:45.457653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5063-9931-4f89-b7ee-dc40606415e4 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:45.458091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d5063-9931-4f89-b7ee-dc40606415e4 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:45.459676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 678/1350] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 11:56:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:45.468946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4be3d6-68e1-4391-b5de-3915589ff939 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:45.469490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4be3d6-68e1-4391-b5de-3915589ff939 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:45.469807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4be3d6-68e1-4391-b5de-3915589ff939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:45.603866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3e4be3d6-68e1-4391-b5de-3915589ff939 None None] MFA Rules not processed for user `9db663d35f964d42938be0d1a9ed4071`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:45.605420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3e4be3d6-68e1-4391-b5de-3915589ff939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:45.632608 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:45.860753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 673/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 11:56:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:51.361415 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa0f31-83f8-4f8d-8e9c-35347ee02dd5 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:51.363136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa0f31-83f8-4f8d-8e9c-35347ee02dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:51.363840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa0f31-83f8-4f8d-8e9c-35347ee02dd5 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:51.366198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 679/1352] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 11:56:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:51.372505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5496de9-b2cd-4313-ac55-e286adc8f4bc None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:51.373452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5496de9-b2cd-4313-ac55-e286adc8f4bc None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:51.373452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5496de9-b2cd-4313-ac55-e286adc8f4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:51.504979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a5496de9-b2cd-4313-ac55-e286adc8f4bc None None] MFA Rules not processed for user `9db663d35f964d42938be0d1a9ed4071`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:51.506525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a5496de9-b2cd-4313-ac55-e286adc8f4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:51.612596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:51.664488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 674/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 11:56:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:56:52.436806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5915ca-de92-4f55-bcc4-1ee623549507 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:52.438512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5915ca-de92-4f55-bcc4-1ee623549507 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:52.439537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5915ca-de92-4f55-bcc4-1ee623549507 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:52.442882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 680/1354] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 11:56:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:52.468870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c357b-8b10-489c-8297-2beba637e611 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:52.472643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c357b-8b10-489c-8297-2beba637e611 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:52.473064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c357b-8b10-489c-8297-2beba637e611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:52.659357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-113c357b-8b10-489c-8297-2beba637e611 None None] MFA Rules not processed for user `9db663d35f964d42938be0d1a9ed4071`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:52.660934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-113c357b-8b10-489c-8297-2beba637e611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:52.878547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 675/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 11:56:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:56:56.156953 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967fdf9-43eb-4d42-8220-480fa43b590c None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:56.157768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967fdf9-43eb-4d42-8220-480fa43b590c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:56.158276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967fdf9-43eb-4d42-8220-480fa43b590c None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:56.160756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 681/1356] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:56:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:56.168728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4c652-7dde-4e9f-8a0a-e616e1834b83 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:56.169639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4c652-7dde-4e9f-8a0a-e616e1834b83 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:56.169639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4c652-7dde-4e9f-8a0a-e616e1834b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:56.230242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:56.288275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-5df4c652-7dde-4e9f-8a0a-e616e1834b83 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:56.290271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5df4c652-7dde-4e9f-8a0a-e616e1834b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:56.456690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 676/1357] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:56.560188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0575-836b-4102-97b5-2a764808bf9f None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:56.561080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0575-836b-4102-97b5-2a764808bf9f None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:56.561799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b0575-836b-4102-97b5-2a764808bf9f None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:56.566142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 682/1358] 10.222.0.49 () {54 vars in 962 bytes} [Mon Aug 1 11:56:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:56:56.590109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf978-7419-425e-8f33-01cf6b4ffde0 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:56.590544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf978-7419-425e-8f33-01cf6b4ffde0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:56.591047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf978-7419-425e-8f33-01cf6b4ffde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:56.741768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c02cf978-7419-425e-8f33-01cf6b4ffde0 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:56:56.742962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c02cf978-7419-425e-8f33-01cf6b4ffde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:56.813048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:56.922230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 677/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Aug 1 11:56:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:56:56.930391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b0575-836b-4102-97b5-2a764808bf9f None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:56:56.931920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b24b0575-836b-4102-97b5-2a764808bf9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:57.128344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b0575-836b-4102-97b5-2a764808bf9f None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:56:57.235998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:56:57.300960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf8deb-ceaf-4ac1-a32c-af2bc4d93cea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:56:57.301972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf8deb-ceaf-4ac1-a32c-af2bc4d93cea service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:56:57.302487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf8deb-ceaf-4ac1-a32c-af2bc4d93cea service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:56:57.302965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf8deb-ceaf-4ac1-a32c-af2bc4d93cea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:56:57.305866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-08bf8deb-ceaf-4ac1-a32c-af2bc4d93cea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:56:57.444373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf8deb-ceaf-4ac1-a32c-af2bc4d93cea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:56:57.445935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf8deb-ceaf-4ac1-a32c-af2bc4d93cea service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:56:57.483070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 683/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:56:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 11:57:00.132003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f8df9-1278-4f09-9ad0-3c0d87f35206 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:00.132905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f8df9-1278-4f09-9ad0-3c0d87f35206 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:00.133466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f8df9-1278-4f09-9ad0-3c0d87f35206 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:00.135779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 678/1361] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:00.142307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf03c1f-1505-4dc9-9a70-14df8ef1575c None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:00.143143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf03c1f-1505-4dc9-9a70-14df8ef1575c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:00.143143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf03c1f-1505-4dc9-9a70-14df8ef1575c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:00.224452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-1cf03c1f-1505-4dc9-9a70-14df8ef1575c None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:00.225695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1cf03c1f-1505-4dc9-9a70-14df8ef1575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:00.273018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:00.360392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 684/1362] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:57:00.440145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f8df9-1278-4f09-9ad0-3c0d87f35206 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:00.443029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-757f8df9-1278-4f09-9ad0-3c0d87f35206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:00.682745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f8df9-1278-4f09-9ad0-3c0d87f35206 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:00.696384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:00.859288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093054d7-bdf3-4d74-ad61-b57917bb593a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:00.860440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093054d7-bdf3-4d74-ad61-b57917bb593a service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:00.861146 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093054d7-bdf3-4d74-ad61-b57917bb593a service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:00.861619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093054d7-bdf3-4d74-ad61-b57917bb593a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:00.864836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-093054d7-bdf3-4d74-ad61-b57917bb593a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:01.041226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093054d7-bdf3-4d74-ad61-b57917bb593a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:01.043600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093054d7-bdf3-4d74-ad61-b57917bb593a service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:01.090939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 679/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:04.068514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2706ce-ac14-433b-a632-f79733a89d7b None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:04.069091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2706ce-ac14-433b-a632-f79733a89d7b None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:04.069091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2706ce-ac14-433b-a632-f79733a89d7b None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:04.070727 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 685/1364] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:57:04.076651 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b643c-f234-4247-9195-f73cdbdd3f2f None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:04.077231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b643c-f234-4247-9195-f73cdbdd3f2f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:04.077231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b643c-f234-4247-9195-f73cdbdd3f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:04.095818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:04.164882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e76b643c-f234-4247-9195-f73cdbdd3f2f None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:04.166109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e76b643c-f234-4247-9195-f73cdbdd3f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:04.354242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 680/1365] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:04.437366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336e577-3a70-43ad-a18a-c66592dd65e6 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:04.437802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336e577-3a70-43ad-a18a-c66592dd65e6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:04.438652 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336e577-3a70-43ad-a18a-c66592dd65e6 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:04.441243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 686/1366] 10.222.0.49 () {54 vars in 962 bytes} [Mon Aug 1 11:57:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:04.448378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86a6da-87a3-472f-9107-506581ac9322 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:04.448811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86a6da-87a3-472f-9107-506581ac9322 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:04.449602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d86a6da-87a3-472f-9107-506581ac9322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:04.572739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-0d86a6da-87a3-472f-9107-506581ac9322 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:04.575189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0d86a6da-87a3-472f-9107-506581ac9322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:04.630633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:04.764360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 681/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Aug 1 11:57:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:04.773376 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336e577-3a70-43ad-a18a-c66592dd65e6 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:04.775751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2336e577-3a70-43ad-a18a-c66592dd65e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:05.108374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336e577-3a70-43ad-a18a-c66592dd65e6 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:05.155897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:05.345247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaef6e0-2589-4030-b6ab-969b065e3988 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:05.346747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaef6e0-2589-4030-b6ab-969b065e3988 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:05.347245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaef6e0-2589-4030-b6ab-969b065e3988 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:05.347728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaef6e0-2589-4030-b6ab-969b065e3988 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:05.351090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bfaef6e0-2589-4030-b6ab-969b065e3988 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:05.537235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaef6e0-2589-4030-b6ab-969b065e3988 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:05.541496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaef6e0-2589-4030-b6ab-969b065e3988 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:05.588668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 687/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 816 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:06.074178 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210add69-b86d-42ec-8132-99c9788f27c0 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:06.075022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210add69-b86d-42ec-8132-99c9788f27c0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:06.075022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210add69-b86d-42ec-8132-99c9788f27c0 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:06.076551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 682/1369] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:57:06.082640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61710011-6aaa-400f-9de0-5428b4de26d3 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:06.082934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61710011-6aaa-400f-9de0-5428b4de26d3 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:06.083189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61710011-6aaa-400f-9de0-5428b4de26d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:06.125473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:06.174280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-61710011-6aaa-400f-9de0-5428b4de26d3 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:06.175571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-61710011-6aaa-400f-9de0-5428b4de26d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:06.318136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 688/1370] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:57:06.651811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21470b1e-70fe-4e88-92bd-0e75601aca39 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:06.652485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21470b1e-70fe-4e88-92bd-0e75601aca39 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:06.653039 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21470b1e-70fe-4e88-92bd-0e75601aca39 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:06.656052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 683/1371] 10.222.0.49 () {54 vars in 964 bytes} [Mon Aug 1 11:57:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:06.665169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3ca30-1aab-44aa-a44f-eca23d31a319 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:06.665663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3ca30-1aab-44aa-a44f-eca23d31a319 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:06.666145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3ca30-1aab-44aa-a44f-eca23d31a319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:06.802488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-cbc3ca30-1aab-44aa-a44f-eca23d31a319 None None] MFA Rules not processed for user `dc0102a256934641a999dd091e6ba0b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:06.803964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-cbc3ca30-1aab-44aa-a44f-eca23d31a319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:06.868920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:06.914803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 689/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Mon Aug 1 11:57:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:06.932182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21470b1e-70fe-4e88-92bd-0e75601aca39 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:06.937724 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-21470b1e-70fe-4e88-92bd-0e75601aca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:07.107077 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21470b1e-70fe-4e88-92bd-0e75601aca39 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:07.162764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:07.212327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e214b207-e5c7-4cee-8c8c-eff2d619d89a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:07.213453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214b207-e5c7-4cee-8c8c-eff2d619d89a service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:07.213922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214b207-e5c7-4cee-8c8c-eff2d619d89a service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:07.214405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e214b207-e5c7-4cee-8c8c-eff2d619d89a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:07.219452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e214b207-e5c7-4cee-8c8c-eff2d619d89a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:07.398494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214b207-e5c7-4cee-8c8c-eff2d619d89a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:07.401663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e214b207-e5c7-4cee-8c8c-eff2d619d89a service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:07.441979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 684/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:08.198151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec749eb5-2830-4895-a64b-d6e3e2267108 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:08.199126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec749eb5-2830-4895-a64b-d6e3e2267108 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:08.199126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec749eb5-2830-4895-a64b-d6e3e2267108 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:08.201250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 690/1374] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:08.209462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eabfc3-2fd5-4a4e-914f-be85f3e2889d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:08.210123 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eabfc3-2fd5-4a4e-914f-be85f3e2889d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:08.210123 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eabfc3-2fd5-4a4e-914f-be85f3e2889d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:08.296989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-28eabfc3-2fd5-4a4e-914f-be85f3e2889d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:08.298282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-28eabfc3-2fd5-4a4e-914f-be85f3e2889d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:08.327746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:08.440056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 685/1375] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:57:08.491305 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec749eb5-2830-4895-a64b-d6e3e2267108 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:08.493100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ec749eb5-2830-4895-a64b-d6e3e2267108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:08.682806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec749eb5-2830-4895-a64b-d6e3e2267108 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:08.759036 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:08.847950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84a0d1-8508-4a77-88f4-03478447d6ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:08.849079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a0d1-8508-4a77-88f4-03478447d6ad service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:08.849569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a0d1-8508-4a77-88f4-03478447d6ad service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:08.850042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a0d1-8508-4a77-88f4-03478447d6ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:08.853323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ac84a0d1-8508-4a77-88f4-03478447d6ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:08.979418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84a0d1-8508-4a77-88f4-03478447d6ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:08.981508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84a0d1-8508-4a77-88f4-03478447d6ad service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:09.005561 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 691/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:11.333137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b563-3f84-46d0-8180-4d4eff37aa33 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:11.362221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b563-3f84-46d0-8180-4d4eff37aa33 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:11.362221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b563-3f84-46d0-8180-4d4eff37aa33 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:11.362221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 686/1377] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:11 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:11.362651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1c5ce-8e2a-4f73-b540-bcf10abe6c25 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:11.362651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1c5ce-8e2a-4f73-b540-bcf10abe6c25 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:11.362651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e1c5ce-8e2a-4f73-b540-bcf10abe6c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:11.487994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-07e1c5ce-8e2a-4f73-b540-bcf10abe6c25 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:11.490545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-07e1c5ce-8e2a-4f73-b540-bcf10abe6c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:11.499863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:11.726273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 692/1378] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:57:11.814334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64b563-3f84-46d0-8180-4d4eff37aa33 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:11.817516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1b64b563-3f84-46d0-8180-4d4eff37aa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:12.055229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64b563-3f84-46d0-8180-4d4eff37aa33 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:12.060044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:12.197319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85374e38-9fee-416a-a3a2-decfa10d284b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:12.198292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85374e38-9fee-416a-a3a2-decfa10d284b service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:12.198731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85374e38-9fee-416a-a3a2-decfa10d284b service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:12.199159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85374e38-9fee-416a-a3a2-decfa10d284b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:12.201465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-85374e38-9fee-416a-a3a2-decfa10d284b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:12.316761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85374e38-9fee-416a-a3a2-decfa10d284b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:12.318493 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85374e38-9fee-416a-a3a2-decfa10d284b service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:12.342117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 687/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:14.879920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c78b0e-36bf-4fba-aab3-540c75d4044e None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:14.880780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c78b0e-36bf-4fba-aab3-540c75d4044e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:14.880780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c78b0e-36bf-4fba-aab3-540c75d4044e None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:14.885970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 693/1380] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:14.889312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc36b3-270f-401a-8e07-4e0c86fed529 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:14.890357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc36b3-270f-401a-8e07-4e0c86fed529 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:14.890713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc36b3-270f-401a-8e07-4e0c86fed529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:14.903625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:14.971170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-cdbc36b3-270f-401a-8e07-4e0c86fed529 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:14.972522 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-cdbc36b3-270f-401a-8e07-4e0c86fed529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:15.099898 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 688/1381] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:15.153055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c78b0e-36bf-4fba-aab3-540c75d4044e None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:15.155605 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b8c78b0e-36bf-4fba-aab3-540c75d4044e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:15.356995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:15.372520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c78b0e-36bf-4fba-aab3-540c75d4044e None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:15.529065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14015c7-3c8e-4bcf-808b-b4b7c5d0cb50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:15.529748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14015c7-3c8e-4bcf-808b-b4b7c5d0cb50 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:15.530024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14015c7-3c8e-4bcf-808b-b4b7c5d0cb50 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:15.530287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14015c7-3c8e-4bcf-808b-b4b7c5d0cb50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:15.532062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f14015c7-3c8e-4bcf-808b-b4b7c5d0cb50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:15.668023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14015c7-3c8e-4bcf-808b-b4b7c5d0cb50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:15.670365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14015c7-3c8e-4bcf-808b-b4b7c5d0cb50 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:15.690317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:15.704079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 694/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 11:57:18.444190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696274c-718f-4c34-aa48-aaa63a1fce20 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:18.445620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696274c-718f-4c34-aa48-aaa63a1fce20 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:18.445620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696274c-718f-4c34-aa48-aaa63a1fce20 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:18.447551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 689/1383] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:18.458576 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca442e7-6b85-4024-a2aa-dca3a3c5dea4 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:18.459217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca442e7-6b85-4024-a2aa-dca3a3c5dea4 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:18.459697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca442e7-6b85-4024-a2aa-dca3a3c5dea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:18.600760 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-eca442e7-6b85-4024-a2aa-dca3a3c5dea4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:18.602652 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-eca442e7-6b85-4024-a2aa-dca3a3c5dea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:18.778196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 695/1384] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:18.828183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696274c-718f-4c34-aa48-aaa63a1fce20 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:18.830473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2696274c-718f-4c34-aa48-aaa63a1fce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:19.002054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:19.130456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696274c-718f-4c34-aa48-aaa63a1fce20 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:19.372715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bbbe9-0f79-4a66-956b-74725b3443a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:19.373935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bbbe9-0f79-4a66-956b-74725b3443a0 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:19.374455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bbbe9-0f79-4a66-956b-74725b3443a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:19.374960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bbbe9-0f79-4a66-956b-74725b3443a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:19.379152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7b8bbbe9-0f79-4a66-956b-74725b3443a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:19.476453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:19.540391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bbbe9-0f79-4a66-956b-74725b3443a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:19.542809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8bbbe9-0f79-4a66-956b-74725b3443a0 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:19.572777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 690/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:22.205735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498c6b-c845-4b39-9ed1-5b82a410ba03 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:22.206345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498c6b-c845-4b39-9ed1-5b82a410ba03 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:22.206699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498c6b-c845-4b39-9ed1-5b82a410ba03 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:22.209388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 696/1386] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:22.219057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b82b1b-24cb-4d68-8ea4-78b3d425c5d6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:22.219749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b82b1b-24cb-4d68-8ea4-78b3d425c5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:22.220403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b82b1b-24cb-4d68-8ea4-78b3d425c5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:22.344107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e6b82b1b-24cb-4d68-8ea4-78b3d425c5d6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:22.345834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e6b82b1b-24cb-4d68-8ea4-78b3d425c5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:22.470499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 691/1387] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:22.524905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f498c6b-c845-4b39-9ed1-5b82a410ba03 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:22.527780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4f498c6b-c845-4b39-9ed1-5b82a410ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:22.657270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:22.785405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f498c6b-c845-4b39-9ed1-5b82a410ba03 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:22.995289 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ead21-ec51-45e8-820e-5d1ff3b9c297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:22.996207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ead21-ec51-45e8-820e-5d1ff3b9c297 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:22.996626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ead21-ec51-45e8-820e-5d1ff3b9c297 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:22.997025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ead21-ec51-45e8-820e-5d1ff3b9c297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:22.999489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b65ead21-ec51-45e8-820e-5d1ff3b9c297 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:23.051185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:23.152102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ead21-ec51-45e8-820e-5d1ff3b9c297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:23.155438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ead21-ec51-45e8-820e-5d1ff3b9c297 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:23.193288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 697/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:24.333562 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5c6d0-8a2c-4427-9ad9-b5aba510af73 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:24.334149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5c6d0-8a2c-4427-9ad9-b5aba510af73 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:24.334149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f5c6d0-8a2c-4427-9ad9-b5aba510af73 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:24.335615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 692/1389] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:24.342838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c64da-c759-45be-aa33-c327e8d0ac87 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:24.343610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c64da-c759-45be-aa33-c327e8d0ac87 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:24.343811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c64da-c759-45be-aa33-c327e8d0ac87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:24.464848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-387c64da-c759-45be-aa33-c327e8d0ac87 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:24.466226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-387c64da-c759-45be-aa33-c327e8d0ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:24.596050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 698/1390] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:24.656024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca031db-e2f8-4dd6-917c-b9f4e22f7638 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:24.656455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca031db-e2f8-4dd6-917c-b9f4e22f7638 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:24.656804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca031db-e2f8-4dd6-917c-b9f4e22f7638 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:24.658265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 693/1391] 10.222.0.49 () {54 vars in 966 bytes} [Mon Aug 1 11:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:24.664467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882628aa-df87-4329-b410-07a3176609d7 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:24.664881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882628aa-df87-4329-b410-07a3176609d7 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:24.665257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882628aa-df87-4329-b410-07a3176609d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:24.684280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:24.758529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-882628aa-df87-4329-b410-07a3176609d7 None None] MFA Rules not processed for user `184005a383ff4ab696d5235369b44628`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:24.759782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-882628aa-df87-4329-b410-07a3176609d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:24.856271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 699/1392] 10.222.0.49 () {58 vars in 1063 bytes} [Mon Aug 1 11:57:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:24.864613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca031db-e2f8-4dd6-917c-b9f4e22f7638 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:24.866540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-bca031db-e2f8-4dd6-917c-b9f4e22f7638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:24.903221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:24.996835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca031db-e2f8-4dd6-917c-b9f4e22f7638 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:25.089386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722394b6-c29e-4931-83dd-98696da4fdef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:25.090125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722394b6-c29e-4931-83dd-98696da4fdef service glance] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:25.090491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722394b6-c29e-4931-83dd-98696da4fdef service glance] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:25.090935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722394b6-c29e-4931-83dd-98696da4fdef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:25.093217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-722394b6-c29e-4931-83dd-98696da4fdef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:25.192526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:25.257221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722394b6-c29e-4931-83dd-98696da4fdef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:25.259744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722394b6-c29e-4931-83dd-98696da4fdef service glance] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:25.296804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 694/1393] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:25.978189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6df47-a333-4c40-8509-8afebc8257a6 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:25.978846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6df47-a333-4c40-8509-8afebc8257a6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:25.979081 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6df47-a333-4c40-8509-8afebc8257a6 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:25.981681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 700/1394] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:25.990446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ac240-dd4b-4a8f-8a7d-642f553664a9 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:25.991350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ac240-dd4b-4a8f-8a7d-642f553664a9 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:25.991350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ac240-dd4b-4a8f-8a7d-642f553664a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:26.072104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3b4ac240-dd4b-4a8f-8a7d-642f553664a9 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:26.073398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3b4ac240-dd4b-4a8f-8a7d-642f553664a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:26.203752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 695/1395] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:57:26.253668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6df47-a333-4c40-8509-8afebc8257a6 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:26.255394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ebf6df47-a333-4c40-8509-8afebc8257a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:26.356108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc1f5-27f1-48c7-b9dd-e4d58b68bf1c None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:26.356515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc1f5-27f1-48c7-b9dd-e4d58b68bf1c None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:26.356807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc1f5-27f1-48c7-b9dd-e4d58b68bf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:26.411642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:26.415785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:26.444714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-412bc1f5-27f1-48c7-b9dd-e4d58b68bf1c None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:26.446332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-412bc1f5-27f1-48c7-b9dd-e4d58b68bf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:26.498199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf6df47-a333-4c40-8509-8afebc8257a6 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:26.567474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 696/1396] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 11:57:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:57:26.623438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca9fda-21f5-4ea0-88bb-759c6057d871 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:26.623749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca9fda-21f5-4ea0-88bb-759c6057d871 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:26.624282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca9fda-21f5-4ea0-88bb-759c6057d871 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:26.627139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 697/1397] 10.222.0.49 () {54 vars in 943 bytes} [Mon Aug 1 11:57:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:26.637960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36f47e-1bdc-4b5e-a5b7-de1985c2665e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:26.638298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36f47e-1bdc-4b5e-a5b7-de1985c2665e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:26.638501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36f47e-1bdc-4b5e-a5b7-de1985c2665e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:26.674585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34263a01-ee92-4087-8fd3-bbe4eb893b70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:26.675218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34263a01-ee92-4087-8fd3-bbe4eb893b70 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:26.677124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34263a01-ee92-4087-8fd3-bbe4eb893b70 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:26.677718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34263a01-ee92-4087-8fd3-bbe4eb893b70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:26.680596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-34263a01-ee92-4087-8fd3-bbe4eb893b70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:26.815626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:26.857613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34263a01-ee92-4087-8fd3-bbe4eb893b70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:26.860918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34263a01-ee92-4087-8fd3-bbe4eb893b70 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:26.885555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ea36f47e-1bdc-4b5e-a5b7-de1985c2665e None None] MFA Rules not processed for user `b9b6a96421e349a88a38590cfce167fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:26.886961 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ea36f47e-1bdc-4b5e-a5b7-de1985c2665e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:26.897945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 701/1398] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 11:57:26.968391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:27.047716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 698/1399] 10.222.0.49 () {58 vars in 1040 bytes} [Mon Aug 1 11:57:26 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:57:27.058937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34263a01-ee92-4087-8fd3-bbe4eb893b70 service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:27.060663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-34263a01-ee92-4087-8fd3-bbe4eb893b70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:27.204878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34263a01-ee92-4087-8fd3-bbe4eb893b70 service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:27.338986 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:27.341296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957f593-4cf3-43c2-8104-e8568ae91127 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:27.342273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957f593-4cf3-43c2-8104-e8568ae91127 service swift] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:27.342649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957f593-4cf3-43c2-8104-e8568ae91127 service swift] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:27.342939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957f593-4cf3-43c2-8104-e8568ae91127 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:27.346045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1957f593-4cf3-43c2-8104-e8568ae91127 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:27.448380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957f593-4cf3-43c2-8104-e8568ae91127 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:27.450102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957f593-4cf3-43c2-8104-e8568ae91127 service swift] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:27.460717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 702/1400] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 11:57:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 11:57:29.487150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719656aa-d750-4e8d-bb60-e6d06b158d41 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:29.487794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719656aa-d750-4e8d-bb60-e6d06b158d41 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:29.488051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719656aa-d750-4e8d-bb60-e6d06b158d41 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:29.493053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 699/1401] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:29.499192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136d4d2-4740-4c65-ae47-22b0d6225377 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:29.500076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136d4d2-4740-4c65-ae47-22b0d6225377 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:29.500691 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136d4d2-4740-4c65-ae47-22b0d6225377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:29.619399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7136d4d2-4740-4c65-ae47-22b0d6225377 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:29.621262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7136d4d2-4740-4c65-ae47-22b0d6225377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:29.797264 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:29.809974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 703/1402] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:29.858091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719656aa-d750-4e8d-bb60-e6d06b158d41 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:29.860045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-719656aa-d750-4e8d-bb60-e6d06b158d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:30.101003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719656aa-d750-4e8d-bb60-e6d06b158d41 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:30.182417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:30.317654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3fa27-23be-4ed1-bf1d-dd5c81f3c244 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:30.318654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3fa27-23be-4ed1-bf1d-dd5c81f3c244 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:30.319208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3fa27-23be-4ed1-bf1d-dd5c81f3c244 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:30.319556 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3fa27-23be-4ed1-bf1d-dd5c81f3c244 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:30.322108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-79a3fa27-23be-4ed1-bf1d-dd5c81f3c244 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:30.514707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3fa27-23be-4ed1-bf1d-dd5c81f3c244 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:30.517191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a3fa27-23be-4ed1-bf1d-dd5c81f3c244 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:30.542445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 700/1403] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 11:57:33.881280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342303f-8a99-4543-aa58-a4d0f906b097 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:33.881988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342303f-8a99-4543-aa58-a4d0f906b097 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:33.882277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342303f-8a99-4543-aa58-a4d0f906b097 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:33.885025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 704/1404] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:33.894640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c29a4-ba6f-46ac-901c-b2f4edac9ae0 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:33.895696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c29a4-ba6f-46ac-901c-b2f4edac9ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:33.896254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c29a4-ba6f-46ac-901c-b2f4edac9ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:33.979145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:33.997278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1d7c29a4-ba6f-46ac-901c-b2f4edac9ae0 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:33.998755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1d7c29a4-ba6f-46ac-901c-b2f4edac9ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:34.189817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 701/1405] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:34.247643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342303f-8a99-4543-aa58-a4d0f906b097 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:34.249329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1342303f-8a99-4543-aa58-a4d0f906b097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:34.519158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342303f-8a99-4543-aa58-a4d0f906b097 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:34.674854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:34.763588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c394da3-fcdd-47e8-a487-afa71745de2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:34.764271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c394da3-fcdd-47e8-a487-afa71745de2b service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:34.764597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c394da3-fcdd-47e8-a487-afa71745de2b service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:34.764973 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c394da3-fcdd-47e8-a487-afa71745de2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:34.766715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9c394da3-fcdd-47e8-a487-afa71745de2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:34.914031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c394da3-fcdd-47e8-a487-afa71745de2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:34.915726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c394da3-fcdd-47e8-a487-afa71745de2b service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:34.939680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 705/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:37.737890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f609d2-b890-473d-b725-5877b231ce12 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:37.738745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f609d2-b890-473d-b725-5877b231ce12 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:37.738896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f609d2-b890-473d-b725-5877b231ce12 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:37.741391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 702/1407] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:57:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:37.747522 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58900981-b4c2-49fa-b948-92031de9ad49 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:37.748394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58900981-b4c2-49fa-b948-92031de9ad49 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:37.748394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58900981-b4c2-49fa-b948-92031de9ad49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:37.839527 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-58900981-b4c2-49fa-b948-92031de9ad49 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:37.840919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-58900981-b4c2-49fa-b948-92031de9ad49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:37.889282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:38.045231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 706/1408] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:57:38.104979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f609d2-b890-473d-b725-5877b231ce12 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:38.108536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d4f609d2-b890-473d-b725-5877b231ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:38.374613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:38.435876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f609d2-b890-473d-b725-5877b231ce12 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:38.640107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f6eb2-27fb-425f-96b4-6abe365ac9b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:38.641602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f6eb2-27fb-425f-96b4-6abe365ac9b9 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:38.642043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f6eb2-27fb-425f-96b4-6abe365ac9b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:38.642533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f6eb2-27fb-425f-96b4-6abe365ac9b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:38.645109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d87f6eb2-27fb-425f-96b4-6abe365ac9b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:38.777235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:38.782331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f6eb2-27fb-425f-96b4-6abe365ac9b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:38.784417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f6eb2-27fb-425f-96b4-6abe365ac9b9 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:38.806611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 703/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:48.027058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=30 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:48.804891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=7 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:51.831467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ef1b8-ec61-40ff-bd65-2afa72dff2ee None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:51.832235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ef1b8-ec61-40ff-bd65-2afa72dff2ee None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:51.832235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ef1b8-ec61-40ff-bd65-2afa72dff2ee None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:51.833754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 707/1410] 10.222.0.49 () {54 vars in 918 bytes} [Mon Aug 1 11:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:51.840963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab587ded-72e8-4750-99d2-122a4ec0e01a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:51.841524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab587ded-72e8-4750-99d2-122a4ec0e01a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:51.841929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab587ded-72e8-4750-99d2-122a4ec0e01a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:51.940194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ab587ded-72e8-4750-99d2-122a4ec0e01a None None] MFA Rules not processed for user `9db663d35f964d42938be0d1a9ed4071`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:51.941377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ab587ded-72e8-4750-99d2-122a4ec0e01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:52.086316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 704/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Mon Aug 1 11:57:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:52.103076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeefe66-27f8-453f-bd47-a8729904f711 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:52.103336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeefe66-27f8-453f-bd47-a8729904f711 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:52.103609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeefe66-27f8-453f-bd47-a8729904f711 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:52.105058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 708/1412] 10.222.0.49 () {54 vars in 918 bytes} [Mon Aug 1 11:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:52.111238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d4ae6-d4b0-4667-8dee-327cc5daa536 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:52.111537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d4ae6-d4b0-4667-8dee-327cc5daa536 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:52.111928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d4ae6-d4b0-4667-8dee-327cc5daa536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:52.200502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:52.204895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b72d4ae6-d4b0-4667-8dee-327cc5daa536 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:52.207285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b72d4ae6-d4b0-4667-8dee-327cc5daa536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:52.422215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 705/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Mon Aug 1 11:57:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:55.029449 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453951f2-de51-4fc9-8825-bcb27f4e916e None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:55.030091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453951f2-de51-4fc9-8825-bcb27f4e916e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:55.030091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453951f2-de51-4fc9-8825-bcb27f4e916e None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:55.031627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:55.038235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc1fa7-2a44-4b83-acbf-1864e93cb006 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:55.038821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc1fa7-2a44-4b83-acbf-1864e93cb006 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:55.038821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc1fa7-2a44-4b83-acbf-1864e93cb006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:55.125742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c2cc1fa7-2a44-4b83-acbf-1864e93cb006 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:55.127022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c2cc1fa7-2a44-4b83-acbf-1864e93cb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:55.189563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:55.241899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:57:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:55.289338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453951f2-de51-4fc9-8825-bcb27f4e916e None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:55.290994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-453951f2-de51-4fc9-8825-bcb27f4e916e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:55.472810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453951f2-de51-4fc9-8825-bcb27f4e916e None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:55.573144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:55.628921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0a553-f7a2-4158-bb3a-b25eb09e55be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:55.629666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0a553-f7a2-4158-bb3a-b25eb09e55be service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:55.629991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0a553-f7a2-4158-bb3a-b25eb09e55be service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:55.630303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0a553-f7a2-4158-bb3a-b25eb09e55be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:55.632123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-eba0a553-f7a2-4158-bb3a-b25eb09e55be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:55.758534 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0a553-f7a2-4158-bb3a-b25eb09e55be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:55.760129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0a553-f7a2-4158-bb3a-b25eb09e55be service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:55.781661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 710/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:56.743999 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d099f-9c75-4b4a-aff2-2a3c60164d47 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:56.744982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d099f-9c75-4b4a-aff2-2a3c60164d47 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:56.744982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d099f-9c75-4b4a-aff2-2a3c60164d47 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:56.746031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 11:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:56.751965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef51683-1761-4795-9649-67f8a78e381a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:56.752719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef51683-1761-4795-9649-67f8a78e381a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:56.752719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef51683-1761-4795-9649-67f8a78e381a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:56.846600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-0ef51683-1761-4795-9649-67f8a78e381a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:56.847867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0ef51683-1761-4795-9649-67f8a78e381a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:56.881817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:56.966510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 11:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:57.121649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d099f-9c75-4b4a-aff2-2a3c60164d47 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:57.123515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d39d099f-9c75-4b4a-aff2-2a3c60164d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:57.300086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d099f-9c75-4b4a-aff2-2a3c60164d47 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:57.350445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:57.442805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee3659-4168-4639-8f56-27a2f1c12eb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:57.443388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee3659-4168-4639-8f56-27a2f1c12eb3 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:57.443762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee3659-4168-4639-8f56-27a2f1c12eb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:57.444078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee3659-4168-4639-8f56-27a2f1c12eb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:57.446292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2eee3659-4168-4639-8f56-27a2f1c12eb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:57.566747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee3659-4168-4639-8f56-27a2f1c12eb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:57.568482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee3659-4168-4639-8f56-27a2f1c12eb3 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:57.590960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 708/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:57:58.806568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f221f86-a8dd-4022-b011-668719afb987 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:58.807196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f221f86-a8dd-4022-b011-668719afb987 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:58.807196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f221f86-a8dd-4022-b011-668719afb987 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:58.809006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 11:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:57:58.814768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d190c-158d-4d99-88e2-6c5f3e82ff63 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:58.815352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d190c-158d-4d99-88e2-6c5f3e82ff63 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:58.815352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d190c-158d-4d99-88e2-6c5f3e82ff63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:58.885913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:58.917659 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-db9d190c-158d-4d99-88e2-6c5f3e82ff63 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:57:58.918939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-db9d190c-158d-4d99-88e2-6c5f3e82ff63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:59.056125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 11:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:57:59.209813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f221f86-a8dd-4022-b011-668719afb987 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:57:59.212614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4f221f86-a8dd-4022-b011-668719afb987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:59.390588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f221f86-a8dd-4022-b011-668719afb987 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:57:59.405420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:57:59.528079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b12d86-892f-446f-84fc-be6cda8350f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:57:59.528963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b12d86-892f-446f-84fc-be6cda8350f1 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:57:59.529352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b12d86-892f-446f-84fc-be6cda8350f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:57:59.529723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b12d86-892f-446f-84fc-be6cda8350f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:57:59.531866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-80b12d86-892f-446f-84fc-be6cda8350f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:57:59.646902 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b12d86-892f-446f-84fc-be6cda8350f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:57:59.648504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b12d86-892f-446f-84fc-be6cda8350f1 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:57:59.671156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 713/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:57:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:00.662414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1688dab-6bef-4d52-997e-345c1fda87f5 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:00.663029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1688dab-6bef-4d52-997e-345c1fda87f5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:00.663335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1688dab-6bef-4d52-997e-345c1fda87f5 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:00.666075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 11:58:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:00.672247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d46864-effe-452d-8f90-a196c73227b8 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:00.672905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d46864-effe-452d-8f90-a196c73227b8 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:00.672905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d46864-effe-452d-8f90-a196c73227b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:00.697068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:00.770861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c9d46864-effe-452d-8f90-a196c73227b8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:00.772101 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c9d46864-effe-452d-8f90-a196c73227b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:00.901369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 11:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:01.056881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1688dab-6bef-4d52-997e-345c1fda87f5 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:01.058520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d1688dab-6bef-4d52-997e-345c1fda87f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:01.193784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:01.236451 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1688dab-6bef-4d52-997e-345c1fda87f5 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:01.374266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969466a-b084-44da-8d1c-7476b8ff75e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:01.374901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969466a-b084-44da-8d1c-7476b8ff75e7 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:01.375191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969466a-b084-44da-8d1c-7476b8ff75e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:01.375439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969466a-b084-44da-8d1c-7476b8ff75e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:01.377232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5969466a-b084-44da-8d1c-7476b8ff75e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:01.514467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:01.541106 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969466a-b084-44da-8d1c-7476b8ff75e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:01.544143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969466a-b084-44da-8d1c-7476b8ff75e7 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:01.579892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 711/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 11:58:02.592917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd155f-2c9e-494c-9a6c-112125436b73 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:02.593698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd155f-2c9e-494c-9a6c-112125436b73 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:02.593698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd155f-2c9e-494c-9a6c-112125436b73 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:02.595375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 11:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:02.601995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5009c7f-6b52-4d9f-9481-a061ea36872f None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:02.602696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5009c7f-6b52-4d9f-9481-a061ea36872f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:02.602696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5009c7f-6b52-4d9f-9481-a061ea36872f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:02.699850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e5009c7f-6b52-4d9f-9481-a061ea36872f None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:02.701170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e5009c7f-6b52-4d9f-9481-a061ea36872f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:02.824394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 11:58:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:58:02.983238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd155f-2c9e-494c-9a6c-112125436b73 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:02.984899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d1fd155f-2c9e-494c-9a6c-112125436b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:03.086947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:03.161182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd155f-2c9e-494c-9a6c-112125436b73 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:03.314937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59be4bc-b8ba-4e75-9ddf-3a68919523f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:03.315660 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59be4bc-b8ba-4e75-9ddf-3a68919523f3 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:03.316027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59be4bc-b8ba-4e75-9ddf-3a68919523f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:03.316438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59be4bc-b8ba-4e75-9ddf-3a68919523f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:03.318296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a59be4bc-b8ba-4e75-9ddf-3a68919523f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:03.381120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:03.431395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59be4bc-b8ba-4e75-9ddf-3a68919523f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:03.433211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59be4bc-b8ba-4e75-9ddf-3a68919523f3 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:03.456841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 716/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:04.499138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f181a61-5c3e-4a60-8aa8-817dcda676be None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:04.499765 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f181a61-5c3e-4a60-8aa8-817dcda676be None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:04.499765 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f181a61-5c3e-4a60-8aa8-817dcda676be None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:04.501501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 11:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:04.507849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a9bcb-1d5b-4198-866b-7f26717666b8 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:04.508686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a9bcb-1d5b-4198-866b-7f26717666b8 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:04.508686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a9bcb-1d5b-4198-866b-7f26717666b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:04.604339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-037a9bcb-1d5b-4198-866b-7f26717666b8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:04.605614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-037a9bcb-1d5b-4198-866b-7f26717666b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:04.761977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 11:58:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:58:04.917930 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f181a61-5c3e-4a60-8aa8-817dcda676be None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:04.919863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2f181a61-5c3e-4a60-8aa8-817dcda676be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:04.980218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:05.155729 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f181a61-5c3e-4a60-8aa8-817dcda676be None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:05.305343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012ee2e-6148-4295-962d-4fc4cf4a6547 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:05.305999 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012ee2e-6148-4295-962d-4fc4cf4a6547 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:05.306266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012ee2e-6148-4295-962d-4fc4cf4a6547 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:05.306536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012ee2e-6148-4295-962d-4fc4cf4a6547 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:05.308272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5012ee2e-6148-4295-962d-4fc4cf4a6547 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:05.317599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:05.428926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012ee2e-6148-4295-962d-4fc4cf4a6547 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:05.431853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012ee2e-6148-4295-962d-4fc4cf4a6547 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:05.463738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 714/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:06.333549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8613025-8772-47b1-935e-4542e63a5394 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:06.334168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8613025-8772-47b1-935e-4542e63a5394 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:06.334168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8613025-8772-47b1-935e-4542e63a5394 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:06.335838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 11:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:06.342030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564763e1-767b-482f-936f-f4e418ff7494 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:06.342723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564763e1-767b-482f-936f-f4e418ff7494 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:06.342723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564763e1-767b-482f-936f-f4e418ff7494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:06.439395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-564763e1-767b-482f-936f-f4e418ff7494 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:06.440851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-564763e1-767b-482f-936f-f4e418ff7494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:06.523143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:06.559211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 11:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:06.709260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8613025-8772-47b1-935e-4542e63a5394 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:06.710811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a8613025-8772-47b1-935e-4542e63a5394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:06.731156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:06.941802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8613025-8772-47b1-935e-4542e63a5394 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:07.048361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:07.082338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf08b47-9d47-45a3-b76d-182ed309ee09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:07.083020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf08b47-9d47-45a3-b76d-182ed309ee09 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:07.083296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf08b47-9d47-45a3-b76d-182ed309ee09 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:07.083566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf08b47-9d47-45a3-b76d-182ed309ee09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:07.085381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4bf08b47-9d47-45a3-b76d-182ed309ee09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:07.220913 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf08b47-9d47-45a3-b76d-182ed309ee09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:07.223563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf08b47-9d47-45a3-b76d-182ed309ee09 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:07.255226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 719/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 11:58:08.134322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12022469-e801-4766-83c2-1e3a8dbbe8e3 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:08.135251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12022469-e801-4766-83c2-1e3a8dbbe8e3 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:08.135251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12022469-e801-4766-83c2-1e3a8dbbe8e3 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:08.137105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 11:58:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:08.142772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d1f28-40b7-4b16-8816-305c16207db2 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:08.143485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d1f28-40b7-4b16-8816-305c16207db2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:08.143485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d1f28-40b7-4b16-8816-305c16207db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:08.238942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-214d1f28-40b7-4b16-8816-305c16207db2 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:08.240430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-214d1f28-40b7-4b16-8816-305c16207db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:08.294799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:08.363762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 11:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:08.516367 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12022469-e801-4766-83c2-1e3a8dbbe8e3 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:08.517891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-12022469-e801-4766-83c2-1e3a8dbbe8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:08.696353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12022469-e801-4766-83c2-1e3a8dbbe8e3 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:08.757600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:08.828867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11536c-8cbe-47bb-86fe-0e9673257c09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:08.829648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11536c-8cbe-47bb-86fe-0e9673257c09 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:08.829960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11536c-8cbe-47bb-86fe-0e9673257c09 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:08.830403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11536c-8cbe-47bb-86fe-0e9673257c09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:08.833454 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ff11536c-8cbe-47bb-86fe-0e9673257c09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:08.941608 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11536c-8cbe-47bb-86fe-0e9673257c09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:08.943266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11536c-8cbe-47bb-86fe-0e9673257c09 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:08.964995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 717/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:09.822190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138eef1-9a4d-4d71-9aff-899520e07efd None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:09.822855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138eef1-9a4d-4d71-9aff-899520e07efd None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:09.822855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138eef1-9a4d-4d71-9aff-899520e07efd None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:09.824236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 11:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:09.831108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb1fb3-c472-48f5-b556-c7855affe854 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:09.831727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb1fb3-c472-48f5-b556-c7855affe854 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:09.832152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb1fb3-c472-48f5-b556-c7855affe854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:09.932226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-36fb1fb3-c472-48f5-b556-c7855affe854 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:09.933744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-36fb1fb3-c472-48f5-b556-c7855affe854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:09.936414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:10.057056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 11:58:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:10.223123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138eef1-9a4d-4d71-9aff-899520e07efd None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:10.224946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4138eef1-9a4d-4d71-9aff-899520e07efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:10.396268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138eef1-9a4d-4d71-9aff-899520e07efd None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:10.429547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:10.539184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be763d-f9e6-464e-b9ed-04decc287183 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:10.539972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be763d-f9e6-464e-b9ed-04decc287183 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:10.540407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be763d-f9e6-464e-b9ed-04decc287183 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:10.540781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be763d-f9e6-464e-b9ed-04decc287183 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:10.542647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e1be763d-f9e6-464e-b9ed-04decc287183 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:10.651195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be763d-f9e6-464e-b9ed-04decc287183 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:10.653078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be763d-f9e6-464e-b9ed-04decc287183 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:10.675245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 722/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 11:58:12.714762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386ba7b-6341-48f0-9208-f923edfabafd None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:12.715436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386ba7b-6341-48f0-9208-f923edfabafd None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:12.715667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386ba7b-6341-48f0-9208-f923edfabafd None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:12.718303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:58:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:12.724720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d877e-d1b6-4ccb-9072-bd2539b60334 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:12.725328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d877e-d1b6-4ccb-9072-bd2539b60334 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:12.725328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d877e-d1b6-4ccb-9072-bd2539b60334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:12.776054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:12.826698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-726d877e-d1b6-4ccb-9072-bd2539b60334 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:12.828973 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-726d877e-d1b6-4ccb-9072-bd2539b60334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:12.971608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:13.035924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386ba7b-6341-48f0-9208-f923edfabafd None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:13.037690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a386ba7b-6341-48f0-9208-f923edfabafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:13.193297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386ba7b-6341-48f0-9208-f923edfabafd None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:13.234207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:13.286020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0fcfa-db5e-4ec8-84a3-d8b641bd7fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:13.286685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0fcfa-db5e-4ec8-84a3-d8b641bd7fec service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:13.286957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0fcfa-db5e-4ec8-84a3-d8b641bd7fec service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:13.287224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0fcfa-db5e-4ec8-84a3-d8b641bd7fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:13.289030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-bca0fcfa-db5e-4ec8-84a3-d8b641bd7fec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:13.423344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0fcfa-db5e-4ec8-84a3-d8b641bd7fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:13.424872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0fcfa-db5e-4ec8-84a3-d8b641bd7fec service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:13.447944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 720/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:15.801025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68cff2-947a-483a-ba8f-34398804a7dc None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:15.801871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68cff2-947a-483a-ba8f-34398804a7dc None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:15.801871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f68cff2-947a-483a-ba8f-34398804a7dc None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:15.803046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:15.810363 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17ec4f-dfd9-41bc-a0ad-9fab6f2d0e18 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:15.811053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17ec4f-dfd9-41bc-a0ad-9fab6f2d0e18 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:15.811372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17ec4f-dfd9-41bc-a0ad-9fab6f2d0e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:15.901808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-da17ec4f-dfd9-41bc-a0ad-9fab6f2d0e18 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:15.903080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-da17ec4f-dfd9-41bc-a0ad-9fab6f2d0e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:15.918433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:16.055122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:16.095409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68cff2-947a-483a-ba8f-34398804a7dc None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:16.097046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2f68cff2-947a-483a-ba8f-34398804a7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:16.223303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f68cff2-947a-483a-ba8f-34398804a7dc None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:16.237386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:16.329123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a4e8b-95bd-416e-8e80-571dece63dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:16.329757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a4e8b-95bd-416e-8e80-571dece63dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:16.330129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a4e8b-95bd-416e-8e80-571dece63dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:16.330514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a4e8b-95bd-416e-8e80-571dece63dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:16.332481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-906a4e8b-95bd-416e-8e80-571dece63dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:16.477227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a4e8b-95bd-416e-8e80-571dece63dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:16.479557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a4e8b-95bd-416e-8e80-571dece63dd8 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:16.506547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 725/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:19.067944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c9a02-2b91-4377-a7a6-fed87d01dbf4 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:19.069014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c9a02-2b91-4377-a7a6-fed87d01dbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:19.069014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c9a02-2b91-4377-a7a6-fed87d01dbf4 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:19.070587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:58:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:19.077120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8054d-c7dd-4bca-af2e-baf27bbc44e9 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:19.077852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8054d-c7dd-4bca-af2e-baf27bbc44e9 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:19.077852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db8054d-c7dd-4bca-af2e-baf27bbc44e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:19.135403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:19.158209 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2db8054d-c7dd-4bca-af2e-baf27bbc44e9 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:19.159472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2db8054d-c7dd-4bca-af2e-baf27bbc44e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:19.298908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:58:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:58:19.339724 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c9a02-2b91-4377-a7a6-fed87d01dbf4 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:19.342503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a39c9a02-2b91-4377-a7a6-fed87d01dbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:19.526685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c9a02-2b91-4377-a7a6-fed87d01dbf4 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:19.591882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:19.650094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a165b68-c632-4f54-9a52-ac9032327a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:19.650736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a165b68-c632-4f54-9a52-ac9032327a78 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:19.651007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a165b68-c632-4f54-9a52-ac9032327a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:19.651272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a165b68-c632-4f54-9a52-ac9032327a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:19.653078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0a165b68-c632-4f54-9a52-ac9032327a78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:19.797338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a165b68-c632-4f54-9a52-ac9032327a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:19.800643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a165b68-c632-4f54-9a52-ac9032327a78 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:19.827184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 723/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:22.111508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbacaf-b7af-4891-bd1f-1a5159599772 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:22.112147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbacaf-b7af-4891-bd1f-1a5159599772 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:22.112147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbacaf-b7af-4891-bd1f-1a5159599772 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:22.113533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:22.120977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd1122-81e4-43ff-9761-5711cfd16a2c None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:22.121533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd1122-81e4-43ff-9761-5711cfd16a2c None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:22.121853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd1122-81e4-43ff-9761-5711cfd16a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:22.220880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b3bd1122-81e4-43ff-9761-5711cfd16a2c None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:22.222178 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b3bd1122-81e4-43ff-9761-5711cfd16a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:22.250561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:22.370560 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:58:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:22.410778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbacaf-b7af-4891-bd1f-1a5159599772 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:22.412679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6abbacaf-b7af-4891-bd1f-1a5159599772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:22.539033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbacaf-b7af-4891-bd1f-1a5159599772 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:22.574742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:22.640684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fe1cb-0ae2-47be-a3b7-3cdb6d32c8ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:22.641489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fe1cb-0ae2-47be-a3b7-3cdb6d32c8ff service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:22.641877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fe1cb-0ae2-47be-a3b7-3cdb6d32c8ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:22.642278 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fe1cb-0ae2-47be-a3b7-3cdb6d32c8ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:22.644222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-5c1fe1cb-0ae2-47be-a3b7-3cdb6d32c8ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:22.775207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fe1cb-0ae2-47be-a3b7-3cdb6d32c8ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:22.777207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fe1cb-0ae2-47be-a3b7-3cdb6d32c8ff service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:22.799648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 728/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:25.006669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f1b53-b373-47ee-88e4-17c8345e5c63 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:25.007245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f1b53-b373-47ee-88e4-17c8345e5c63 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:25.007425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f1b53-b373-47ee-88e4-17c8345e5c63 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:25.008955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:25.016183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22eb87-97b1-4b3a-8e1c-e458e18d96e4 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:25.016800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22eb87-97b1-4b3a-8e1c-e458e18d96e4 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:25.017171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22eb87-97b1-4b3a-8e1c-e458e18d96e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:25.138023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:25.143944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6a22eb87-97b1-4b3a-8e1c-e458e18d96e4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:25.146558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6a22eb87-97b1-4b3a-8e1c-e458e18d96e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:25.266281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:58:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:25.306394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f1b53-b373-47ee-88e4-17c8345e5c63 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:25.308121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8c2f1b53-b373-47ee-88e4-17c8345e5c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:25.437669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f1b53-b373-47ee-88e4-17c8345e5c63 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:25.497722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:25.527866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e038d-768b-4320-a1ed-c1c1cd6b5d84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:25.528641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e038d-768b-4320-a1ed-c1c1cd6b5d84 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:25.529040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e038d-768b-4320-a1ed-c1c1cd6b5d84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:25.529373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e038d-768b-4320-a1ed-c1c1cd6b5d84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:25.531194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f75e038d-768b-4320-a1ed-c1c1cd6b5d84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:25.673296 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e038d-768b-4320-a1ed-c1c1cd6b5d84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:25.674892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e038d-768b-4320-a1ed-c1c1cd6b5d84 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:25.697721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 726/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:27.855915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55453ff0-da02-4dfd-b72d-99ef450a34c5 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:27.856588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55453ff0-da02-4dfd-b72d-99ef450a34c5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:27.856899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55453ff0-da02-4dfd-b72d-99ef450a34c5 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:27.859806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:58:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:27.866429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec61e62-73e5-4cee-a2e7-623ce11c0243 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:27.867004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec61e62-73e5-4cee-a2e7-623ce11c0243 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:27.867004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec61e62-73e5-4cee-a2e7-623ce11c0243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:27.953611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-fec61e62-73e5-4cee-a2e7-623ce11c0243 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:27.954902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fec61e62-73e5-4cee-a2e7-623ce11c0243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:27.990121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:28.074785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:28.114768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55453ff0-da02-4dfd-b72d-99ef450a34c5 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:28.116504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-55453ff0-da02-4dfd-b72d-99ef450a34c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:28.243894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55453ff0-da02-4dfd-b72d-99ef450a34c5 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:28.281654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:28.335115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dbefd-20cd-40b7-a0b5-81ba52f116e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:28.335730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dbefd-20cd-40b7-a0b5-81ba52f116e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:28.336008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dbefd-20cd-40b7-a0b5-81ba52f116e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:28.336275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dbefd-20cd-40b7-a0b5-81ba52f116e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:28.338244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-657dbefd-20cd-40b7-a0b5-81ba52f116e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:28.469195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657dbefd-20cd-40b7-a0b5-81ba52f116e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:28.470915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657dbefd-20cd-40b7-a0b5-81ba52f116e1 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:28.494249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 731/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:30.718056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff659-99f8-4ddb-92db-aa4647573ee5 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:30.718744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff659-99f8-4ddb-92db-aa4647573ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:30.718744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ff659-99f8-4ddb-92db-aa4647573ee5 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:30.720037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:30.730149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371b6f3-4ad2-48a7-9b53-d4a86e47b63d None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:30.730833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371b6f3-4ad2-48a7-9b53-d4a86e47b63d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:30.731054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371b6f3-4ad2-48a7-9b53-d4a86e47b63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:30.842242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6371b6f3-4ad2-48a7-9b53-d4a86e47b63d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:30.843508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6371b6f3-4ad2-48a7-9b53-d4a86e47b63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:30.854743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:30.985774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:31.026179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff659-99f8-4ddb-92db-aa4647573ee5 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:31.027757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fa3ff659-99f8-4ddb-92db-aa4647573ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:31.152001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ff659-99f8-4ddb-92db-aa4647573ee5 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:31.219787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:31.242908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ad8a3-88a2-439d-b85b-5fc37071f484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:31.243458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ad8a3-88a2-439d-b85b-5fc37071f484 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:31.243738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ad8a3-88a2-439d-b85b-5fc37071f484 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:31.244015 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ad8a3-88a2-439d-b85b-5fc37071f484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:31.245903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-4d7ad8a3-88a2-439d-b85b-5fc37071f484 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:31.427433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ad8a3-88a2-439d-b85b-5fc37071f484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:31.430144 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ad8a3-88a2-439d-b85b-5fc37071f484 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:31.460991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 729/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:33.742122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b65e8-305a-456d-9004-0d6c6f5201bc None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:33.742777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b65e8-305a-456d-9004-0d6c6f5201bc None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:33.742777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b65e8-305a-456d-9004-0d6c6f5201bc None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:33.744054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:58:33.750383 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fece074-fc66-4a1f-919b-1e5614b45caa None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:33.750991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fece074-fc66-4a1f-919b-1e5614b45caa None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:33.750991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fece074-fc66-4a1f-919b-1e5614b45caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:33.838833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-8fece074-fc66-4a1f-919b-1e5614b45caa None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:33.840210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8fece074-fc66-4a1f-919b-1e5614b45caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:33.887362 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:33.960168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:33.999374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b65e8-305a-456d-9004-0d6c6f5201bc None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:34.001015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-520b65e8-305a-456d-9004-0d6c6f5201bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:34.136620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b65e8-305a-456d-9004-0d6c6f5201bc None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:34.184009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:34.228397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382570d3-31b6-4a00-b71b-53e7d3e6be15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:34.229546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382570d3-31b6-4a00-b71b-53e7d3e6be15 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:34.229875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382570d3-31b6-4a00-b71b-53e7d3e6be15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:34.230195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382570d3-31b6-4a00-b71b-53e7d3e6be15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:34.232045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-382570d3-31b6-4a00-b71b-53e7d3e6be15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:34.364851 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382570d3-31b6-4a00-b71b-53e7d3e6be15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:34.366430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382570d3-31b6-4a00-b71b-53e7d3e6be15 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:34.388937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 734/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 11:58:36.645821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e34d54-0e5b-4549-be0b-2054a60114e9 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:36.646495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e34d54-0e5b-4549-be0b-2054a60114e9 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:36.646739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e34d54-0e5b-4549-be0b-2054a60114e9 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:36.649394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:58:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:58:36.656657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218299c1-2e19-415f-a9bc-fd740c59dcd9 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:36.657250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218299c1-2e19-415f-a9bc-fd740c59dcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:36.657517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218299c1-2e19-415f-a9bc-fd740c59dcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:36.759812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-218299c1-2e19-415f-a9bc-fd740c59dcd9 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:36.761148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-218299c1-2e19-415f-a9bc-fd740c59dcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:36.785111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:36.887755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:36.927093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e34d54-0e5b-4549-be0b-2054a60114e9 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:36.928791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a9e34d54-0e5b-4549-be0b-2054a60114e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:37.073897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e34d54-0e5b-4549-be0b-2054a60114e9 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:37.185910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:37.199983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f368a-4b4d-49d4-b0b7-9180b8fad682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:37.200688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f368a-4b4d-49d4-b0b7-9180b8fad682 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:37.200973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f368a-4b4d-49d4-b0b7-9180b8fad682 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:37.201249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f368a-4b4d-49d4-b0b7-9180b8fad682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:37.203117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-da1f368a-4b4d-49d4-b0b7-9180b8fad682 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:37.345700 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f368a-4b4d-49d4-b0b7-9180b8fad682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:37.347398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f368a-4b4d-49d4-b0b7-9180b8fad682 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:37.370514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 732/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:58:39.526329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340150e-6fe6-4f9e-8921-0c5204739420 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:39.527300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340150e-6fe6-4f9e-8921-0c5204739420 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:39.527300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340150e-6fe6-4f9e-8921-0c5204739420 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:39.528328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 11:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:58:39.535226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45bde8-4e4e-45d0-9b0e-30d174913701 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:39.536116 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45bde8-4e4e-45d0-9b0e-30d174913701 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:39.536116 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f45bde8-4e4e-45d0-9b0e-30d174913701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:39.636498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1f45bde8-4e4e-45d0-9b0e-30d174913701 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:58:39.638532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1f45bde8-4e4e-45d0-9b0e-30d174913701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:39.713622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:39.777348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 11:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:58:39.815782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340150e-6fe6-4f9e-8921-0c5204739420 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:58:39.817448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-5340150e-6fe6-4f9e-8921-0c5204739420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:39.955598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340150e-6fe6-4f9e-8921-0c5204739420 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:58:40.013922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:40.046500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7ae23-da08-4283-ac44-c95f41c88c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:58:40.047053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7ae23-da08-4283-ac44-c95f41c88c18 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:58:40.047338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7ae23-da08-4283-ac44-c95f41c88c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:58:40.047610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7ae23-da08-4283-ac44-c95f41c88c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:58:40.049430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d6f7ae23-da08-4283-ac44-c95f41c88c18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:58:40.185159 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7ae23-da08-4283-ac44-c95f41c88c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:58:40.186778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7ae23-da08-4283-ac44-c95f41c88c18 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:58:40.208708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 737/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:58:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 11:58:49.775330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=22 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:58:50.205828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=66 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:01.316372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681fba0-04ee-4b5e-b441-d2cd10c9fe2f None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:01.316968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681fba0-04ee-4b5e-b441-d2cd10c9fe2f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:01.316968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681fba0-04ee-4b5e-b441-d2cd10c9fe2f None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:01.318482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 734/1471] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:59:01.325121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dc1c3-5346-431e-839b-6ddea6057347 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:01.326047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dc1c3-5346-431e-839b-6ddea6057347 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:01.326047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dc1c3-5346-431e-839b-6ddea6057347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:01.533676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-db1dc1c3-5346-431e-839b-6ddea6057347 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:59:01.534955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-db1dc1c3-5346-431e-839b-6ddea6057347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:01.649609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 738/1472] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:59:01.744118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337d905-f6e3-46d9-858f-e79bf4173e22 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:01.744508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337d905-f6e3-46d9-858f-e79bf4173e22 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:01.744836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337d905-f6e3-46d9-858f-e79bf4173e22 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:01.746308 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 735/1473] 10.222.0.49 () {54 vars in 966 bytes} [Mon Aug 1 11:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:59:01.752189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355cd22-087d-4633-82e7-9be5c1a339da None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:01.752586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355cd22-087d-4633-82e7-9be5c1a339da None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:01.752880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355cd22-087d-4633-82e7-9be5c1a339da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:01.846776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9355cd22-087d-4633-82e7-9be5c1a339da None None] MFA Rules not processed for user `184005a383ff4ab696d5235369b44628`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:59:01.848184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9355cd22-087d-4633-82e7-9be5c1a339da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:01.876439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:01.946865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 739/1474] 10.222.0.49 () {58 vars in 1063 bytes} [Mon Aug 1 11:59:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:59:01.960627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337d905-f6e3-46d9-858f-e79bf4173e22 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:59:01.963436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-4337d905-f6e3-46d9-858f-e79bf4173e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:02.102601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337d905-f6e3-46d9-858f-e79bf4173e22 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:59:02.201487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d5bb3a-2258-4724-8c63-26085ccaeeb5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:59:02.202048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5bb3a-2258-4724-8c63-26085ccaeeb5 service glance] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:02.202327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5bb3a-2258-4724-8c63-26085ccaeeb5 service glance] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:02.202600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d5bb3a-2258-4724-8c63-26085ccaeeb5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:02.204526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f0d5bb3a-2258-4724-8c63-26085ccaeeb5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:02.277963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:02.348262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5bb3a-2258-4724-8c63-26085ccaeeb5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:59:02.350132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d5bb3a-2258-4724-8c63-26085ccaeeb5 service glance] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:59:02.375541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 736/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:59:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:59:04.578834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d802e69-8332-4d2c-83c9-cff6ebcd7a5a None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:04.579513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d802e69-8332-4d2c-83c9-cff6ebcd7a5a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:04.579513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d802e69-8332-4d2c-83c9-cff6ebcd7a5a None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:04.580833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 740/1476] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:59:04.587100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c03090-7838-484c-a817-032f9f0c5922 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:04.587740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c03090-7838-484c-a817-032f9f0c5922 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:04.588022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c03090-7838-484c-a817-032f9f0c5922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:04.684254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-44c03090-7838-484c-a817-032f9f0c5922 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:59:04.686411 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-44c03090-7838-484c-a817-032f9f0c5922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:04.813749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 737/1477] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:59:04.845541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d802e69-8332-4d2c-83c9-cff6ebcd7a5a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:59:04.847714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-5d802e69-8332-4d2c-83c9-cff6ebcd7a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:05.003996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d802e69-8332-4d2c-83c9-cff6ebcd7a5a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:59:05.067079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:05.120777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13cbee8-5747-4f58-ae01-33e12253b749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:59:05.121400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cbee8-5747-4f58-ae01-33e12253b749 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:05.121684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cbee8-5747-4f58-ae01-33e12253b749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:05.121959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cbee8-5747-4f58-ae01-33e12253b749 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:05.122678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cbee8-5747-4f58-ae01-33e12253b749 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:59:05.124572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WARNING py.warnings [None req-c13cbee8-5747-4f58-ae01-33e12253b749 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 11:59:05.124572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: warnings.warn(msg) Aug 01 11:59:05.124572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: Aug 01 11:59:05.125727 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13cbee8-5747-4f58-ae01-33e12253b749 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:59:05.148153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 741/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Aug 1 11:59:04 2022] GET /identity/v3/projects => generated 2553 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 11:59:07.273998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b3ea-738b-44b9-a5f6-c8c4b9e62ec3 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:07.274565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b3ea-738b-44b9-a5f6-c8c4b9e62ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:07.274958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b3ea-738b-44b9-a5f6-c8c4b9e62ec3 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:07.276691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 738/1479] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:59:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:59:07.282959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c76c9-bd87-44ce-93bb-ddc77e3dc1f6 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:07.283563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c76c9-bd87-44ce-93bb-ddc77e3dc1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:07.283563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c76c9-bd87-44ce-93bb-ddc77e3dc1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:07.368521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-706c76c9-bd87-44ce-93bb-ddc77e3dc1f6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:59:07.369786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-706c76c9-bd87-44ce-93bb-ddc77e3dc1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:07.476518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 742/1480] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:59:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:59:07.523355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b3ea-738b-44b9-a5f6-c8c4b9e62ec3 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:59:07.526301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8847b3ea-738b-44b9-a5f6-c8c4b9e62ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:07.539789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:07.722882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b3ea-738b-44b9-a5f6-c8c4b9e62ec3 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:59:07.817257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:07.871159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dd2c08-38d5-48dd-9220-63980ead79a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:59:07.871924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd2c08-38d5-48dd-9220-63980ead79a9 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:07.872419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd2c08-38d5-48dd-9220-63980ead79a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:07.872858 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dd2c08-38d5-48dd-9220-63980ead79a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:07.874657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-75dd2c08-38d5-48dd-9220-63980ead79a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:07.971847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd2c08-38d5-48dd-9220-63980ead79a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:59:07.973505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dd2c08-38d5-48dd-9220-63980ead79a9 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:59:07.994346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 739/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:59:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:59:10.379407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f156724-145a-4da6-8926-1ac92c9209c2 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:10.380358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f156724-145a-4da6-8926-1ac92c9209c2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:10.380358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f156724-145a-4da6-8926-1ac92c9209c2 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:10.382029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 743/1482] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:59:10.388677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40733e-536d-4575-a3a9-4b0589b96ea6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:10.389491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40733e-536d-4575-a3a9-4b0589b96ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:10.389491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40733e-536d-4575-a3a9-4b0589b96ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:10.473114 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ee40733e-536d-4575-a3a9-4b0589b96ea6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:59:10.474520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ee40733e-536d-4575-a3a9-4b0589b96ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:10.553880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:10.617401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 740/1483] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 11:59:10.662646 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f156724-145a-4da6-8926-1ac92c9209c2 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:59:10.664365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6f156724-145a-4da6-8926-1ac92c9209c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:10.694531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:10.793662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f156724-145a-4da6-8926-1ac92c9209c2 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:59:10.885203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e293b80-381b-458e-9c02-9a4d2291bb04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:59:10.885905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e293b80-381b-458e-9c02-9a4d2291bb04 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:10.886232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e293b80-381b-458e-9c02-9a4d2291bb04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:10.886556 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e293b80-381b-458e-9c02-9a4d2291bb04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:10.888425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2e293b80-381b-458e-9c02-9a4d2291bb04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:10.988681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:11.031878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e293b80-381b-458e-9c02-9a4d2291bb04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:59:11.033385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e293b80-381b-458e-9c02-9a4d2291bb04 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:59:11.055360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 744/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:59:13.624677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3aaaa-e00f-496a-a1d8-1e59d730e3d5 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:13.625330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3aaaa-e00f-496a-a1d8-1e59d730e3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:13.625330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3aaaa-e00f-496a-a1d8-1e59d730e3d5 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:13.626632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 741/1485] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 11:59:13.633026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a890f68-3e47-443d-8b90-9d45dfb6ad33 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:13.634026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a890f68-3e47-443d-8b90-9d45dfb6ad33 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:13.634026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a890f68-3e47-443d-8b90-9d45dfb6ad33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:13.717508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4a890f68-3e47-443d-8b90-9d45dfb6ad33 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:59:13.718799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4a890f68-3e47-443d-8b90-9d45dfb6ad33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:13.846381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 745/1486] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:59:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:59:13.887181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3aaaa-e00f-496a-a1d8-1e59d730e3d5 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:59:13.889102 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ffc3aaaa-e00f-496a-a1d8-1e59d730e3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:14.018777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3aaaa-e00f-496a-a1d8-1e59d730e3d5 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:59:14.110168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:14.112674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf179583-e374-4f26-9909-0000dbfb491e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:59:14.113363 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf179583-e374-4f26-9909-0000dbfb491e service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:14.113876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf179583-e374-4f26-9909-0000dbfb491e service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:14.114341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf179583-e374-4f26-9909-0000dbfb491e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:14.116473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-bf179583-e374-4f26-9909-0000dbfb491e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:14.256516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf179583-e374-4f26-9909-0000dbfb491e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:59:14.257915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf179583-e374-4f26-9909-0000dbfb491e service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:59:14.280129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 742/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:59:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:59:16.593881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2e789-c747-4331-9088-48f58a12f064 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:16.594471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2e789-c747-4331-9088-48f58a12f064 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:16.594471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2e789-c747-4331-9088-48f58a12f064 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:16.595772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 746/1488] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:59:16.601946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f3fcf-80a8-4882-92bf-65698e64dc23 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:16.602551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f3fcf-80a8-4882-92bf-65698e64dc23 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:16.602551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f3fcf-80a8-4882-92bf-65698e64dc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:16.697148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-8d9f3fcf-80a8-4882-92bf-65698e64dc23 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:59:16.698406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8d9f3fcf-80a8-4882-92bf-65698e64dc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:16.770677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:16.825979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 743/1489] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:59:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:59:16.890893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2e789-c747-4331-9088-48f58a12f064 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:59:16.893708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-72b2e789-c747-4331-9088-48f58a12f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:16.950838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:17.044610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2e789-c747-4331-9088-48f58a12f064 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:59:17.134591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb59b2-6dbc-49fc-9844-dfd5ac64c238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:59:17.135190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb59b2-6dbc-49fc-9844-dfd5ac64c238 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:17.135475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb59b2-6dbc-49fc-9844-dfd5ac64c238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:17.135751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb59b2-6dbc-49fc-9844-dfd5ac64c238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:17.137629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d6bb59b2-6dbc-49fc-9844-dfd5ac64c238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:17.244386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:17.288677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb59b2-6dbc-49fc-9844-dfd5ac64c238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:59:17.291624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb59b2-6dbc-49fc-9844-dfd5ac64c238 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:59:17.329399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 747/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:59:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:59:19.734555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b89083-27fd-4873-ae31-37853d054732 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:19.735238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b89083-27fd-4873-ae31-37853d054732 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:19.735238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b89083-27fd-4873-ae31-37853d054732 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:19.736587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 744/1491] 10.222.0.49 () {54 vars in 926 bytes} [Mon Aug 1 11:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 11:59:19.743812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb4aab-7b16-46db-8aa2-9c57da1844de None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:19.744436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb4aab-7b16-46db-8aa2-9c57da1844de None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:19.744811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb4aab-7b16-46db-8aa2-9c57da1844de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:19.877472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-21eb4aab-7b16-46db-8aa2-9c57da1844de None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 11:59:19.879520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-21eb4aab-7b16-46db-8aa2-9c57da1844de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:20.068575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 748/1492] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Aug 1 11:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 11:59:20.112785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b89083-27fd-4873-ae31-37853d054732 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 11:59:20.114517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-85b89083-27fd-4873-ae31-37853d054732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:20.298593 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b89083-27fd-4873-ae31-37853d054732 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 11:59:20.436394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04735e1f-5e9e-40b8-a5af-bc9746eaf5df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 11:59:20.437182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04735e1f-5e9e-40b8-a5af-bc9746eaf5df service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 11:59:20.437525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04735e1f-5e9e-40b8-a5af-bc9746eaf5df service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 11:59:20.437874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04735e1f-5e9e-40b8-a5af-bc9746eaf5df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 11:59:20.439694 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-04735e1f-5e9e-40b8-a5af-bc9746eaf5df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 11:59:20.451335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:20.616142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04735e1f-5e9e-40b8-a5af-bc9746eaf5df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 11:59:20.617689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04735e1f-5e9e-40b8-a5af-bc9746eaf5df service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 11:59:20.640094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 745/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 11:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 11:59:30.064139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=82 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 11:59:30.637564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=50 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:00:55.584569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688214f-70e9-41cd-8a74-b1c46088dbb0 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:55.585200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688214f-70e9-41cd-8a74-b1c46088dbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:55.585200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688214f-70e9-41cd-8a74-b1c46088dbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:55.687629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c688214f-70e9-41cd-8a74-b1c46088dbb0 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:00:55.689007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c688214f-70e9-41cd-8a74-b1c46088dbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:00:55.821107 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 749/1494] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:00:55.829956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff60c7-f8a5-4dfa-aaec-694f9a807545 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:55.830735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff60c7-f8a5-4dfa-aaec-694f9a807545 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:55.830735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff60c7-f8a5-4dfa-aaec-694f9a807545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:55.863205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-efff60c7-f8a5-4dfa-aaec-694f9a807545 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:00:55.865464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-efff60c7-f8a5-4dfa-aaec-694f9a807545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:00:55.880021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 746/1495] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:00:55.887292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688214f-70e9-41cd-8a74-b1c46088dbb0 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:00:55.898244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688214f-70e9-41cd-8a74-b1c46088dbb0 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:00:55.903832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2c131-199c-46cd-9e18-621ee3d93818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:00:55.904665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2c131-199c-46cd-9e18-621ee3d93818 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:55.904978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2c131-199c-46cd-9e18-621ee3d93818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:55.905316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e2c131-199c-46cd-9e18-621ee3d93818 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:55.906191 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2c131-199c-46cd-9e18-621ee3d93818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:00:55.906602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2c131-199c-46cd-9e18-621ee3d93818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:00:55.907768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WARNING py.warnings [None req-99e2c131-199c-46cd-9e18-621ee3d93818 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:00:55.907768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: warnings.warn(msg) Aug 01 12:00:55.907768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: Aug 01 12:00:55.908968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e2c131-199c-46cd-9e18-621ee3d93818 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:00:55.923025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 750/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:00:55.932026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff60c7-f8a5-4dfa-aaec-694f9a807545 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:00:55.942926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff60c7-f8a5-4dfa-aaec-694f9a807545 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:00:55.949003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b36770-2b79-4f6a-a536-70b98b6f7e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:00:55.949662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b36770-2b79-4f6a-a536-70b98b6f7e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:55.949940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b36770-2b79-4f6a-a536-70b98b6f7e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:55.950206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b36770-2b79-4f6a-a536-70b98b6f7e56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:55.950963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b36770-2b79-4f6a-a536-70b98b6f7e56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:00:55.952892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: WARNING py.warnings [None req-04b36770-2b79-4f6a-a536-70b98b6f7e56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:00:55.952892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: warnings.warn(msg) Aug 01 12:00:55.952892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: Aug 01 12:00:55.954139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b36770-2b79-4f6a-a536-70b98b6f7e56 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:00:55.999057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 747/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:00:55 2022] POST /identity/v3/projects => generated 379 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:00:56.003715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2c131-199c-46cd-9e18-621ee3d93818 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:00:56.015437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e2c131-199c-46cd-9e18-621ee3d93818 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:00:56.076651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:00:56.085060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852a84e-2429-45a7-9346-06c550343c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:00:56.085740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852a84e-2429-45a7-9346-06c550343c81 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.086018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852a84e-2429-45a7-9346-06c550343c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.086288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852a84e-2429-45a7-9346-06c550343c81 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.086976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852a84e-2429-45a7-9346-06c550343c81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:00:56.088073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WARNING py.warnings [None req-8852a84e-2429-45a7-9346-06c550343c81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:00:56.088073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: warnings.warn(msg) Aug 01 12:00:56.088073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: Aug 01 12:00:56.089356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852a84e-2429-45a7-9346-06c550343c81 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:00:56.114749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 751/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:00:56 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:00:56.121007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b36770-2b79-4f6a-a536-70b98b6f7e56 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:00:56.136517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b36770-2b79-4f6a-a536-70b98b6f7e56 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:00:56.143128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78dc19-f877-4b6b-a1c0-1de787938c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:00:56.143847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78dc19-f877-4b6b-a1c0-1de787938c4e admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.144146 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78dc19-f877-4b6b-a1c0-1de787938c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.144473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78dc19-f877-4b6b-a1c0-1de787938c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.145353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78dc19-f877-4b6b-a1c0-1de787938c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:00:56.146419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: WARNING py.warnings [None req-dd78dc19-f877-4b6b-a1c0-1de787938c4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:00:56.146419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: warnings.warn(msg) Aug 01 12:00:56.146419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: Aug 01 12:00:56.147331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78dc19-f877-4b6b-a1c0-1de787938c4e admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:00:56.162392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 748/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:00:56.166946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852a84e-2429-45a7-9346-06c550343c81 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:00:56.178326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852a84e-2429-45a7-9346-06c550343c81 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:00:56.183806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5853f9c-27e6-4273-94b9-0fcdf760191e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:00:56.184717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853f9c-27e6-4273-94b9-0fcdf760191e admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.185046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853f9c-27e6-4273-94b9-0fcdf760191e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.185452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853f9c-27e6-4273-94b9-0fcdf760191e admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.186305 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5853f9c-27e6-4273-94b9-0fcdf760191e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:00:56.187540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WARNING py.warnings [None req-a5853f9c-27e6-4273-94b9-0fcdf760191e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:00:56.187540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: warnings.warn(msg) Aug 01 12:00:56.187540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: Aug 01 12:00:56.188694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5853f9c-27e6-4273-94b9-0fcdf760191e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:00:56.210082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 752/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:00:56.214915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78dc19-f877-4b6b-a1c0-1de787938c4e admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:00:56.225854 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78dc19-f877-4b6b-a1c0-1de787938c4e admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:00:56.230988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3607630a-33c9-40ac-b8f8-28ac11c2cc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:00:56.231841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607630a-33c9-40ac-b8f8-28ac11c2cc95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.232160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607630a-33c9-40ac-b8f8-28ac11c2cc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.232509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3607630a-33c9-40ac-b8f8-28ac11c2cc95 admin admin] PATH_INFO: `/v3/projects/5a68455333574c789b72235d31b7fef2/users/981f877ab34246c78b7dd24180359c78/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.251821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607630a-33c9-40ac-b8f8-28ac11c2cc95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a68455333574c789b72235d31b7fef2, user_id=981f877ab34246c78b7dd24180359c78, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:00:56.252993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: WARNING py.warnings [None req-3607630a-33c9-40ac-b8f8-28ac11c2cc95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:00:56.252993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: warnings.warn(msg) Aug 01 12:00:56.252993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: Aug 01 12:00:56.255183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3607630a-33c9-40ac-b8f8-28ac11c2cc95 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:00:56.271911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 749/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:00:56 2022] PUT /identity/v3/projects/5a68455333574c789b72235d31b7fef2/users/981f877ab34246c78b7dd24180359c78/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:00:56.285797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3123bbb-317a-4a47-a945-760d4a1b18de None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.286331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3123bbb-317a-4a47-a945-760d4a1b18de None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.286820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3123bbb-317a-4a47-a945-760d4a1b18de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.335871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-e3123bbb-317a-4a47-a945-760d4a1b18de None None] MFA Rules not processed for user `981f877ab34246c78b7dd24180359c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:00:56.337923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e3123bbb-317a-4a47-a945-760d4a1b18de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:00:56.425598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 753/1502] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:00:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:00:56.441420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24d2d7-8fc4-40d0-93c2-9e887f6a307c None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.441875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24d2d7-8fc4-40d0-93c2-9e887f6a307c None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.442256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24d2d7-8fc4-40d0-93c2-9e887f6a307c None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.443885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 750/1503] 10.222.0.49 () {54 vars in 964 bytes} [Mon Aug 1 12:00:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:00:56.449955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e62ef1-c927-4ba7-b637-b0d4c7a0bcd2 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.450309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e62ef1-c927-4ba7-b637-b0d4c7a0bcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.450592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e62ef1-c927-4ba7-b637-b0d4c7a0bcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.529335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d0e62ef1-c927-4ba7-b637-b0d4c7a0bcd2 None None] MFA Rules not processed for user `dc0102a256934641a999dd091e6ba0b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:00:56.530600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d0e62ef1-c927-4ba7-b637-b0d4c7a0bcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:00:56.623357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 754/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Mon Aug 1 12:00:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:00:56.632370 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24d2d7-8fc4-40d0-93c2-9e887f6a307c None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:00:56.646448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24d2d7-8fc4-40d0-93c2-9e887f6a307c None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:00:56.653118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611254eb-1426-483f-b1c0-848c661731e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:00:56.653118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611254eb-1426-483f-b1c0-848c661731e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.653690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611254eb-1426-483f-b1c0-848c661731e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.653920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611254eb-1426-483f-b1c0-848c661731e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.666684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611254eb-1426-483f-b1c0-848c661731e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:00:56.669577 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611254eb-1426-483f-b1c0-848c661731e8 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:00:56.684004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 751/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:00:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:00:56.734540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02a8c3-41fd-4d56-80c0-5f41ba6c00b3 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.734846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02a8c3-41fd-4d56-80c0-5f41ba6c00b3 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.735084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02a8c3-41fd-4d56-80c0-5f41ba6c00b3 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.736476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 755/1506] 10.222.0.49 () {52 vars in 830 bytes} [Mon Aug 1 12:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:00:56.741272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611254eb-1426-483f-b1c0-848c661731e8 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:00:56.812626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611254eb-1426-483f-b1c0-848c661731e8 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:00:56.817625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:00:56.818296 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.818576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.818850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] PATH_INFO: `/v3/users/981f877ab34246c78b7dd24180359c78` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.820079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=981f877ab34246c78b7dd24180359c78)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:00:56.821175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: WARNING py.warnings [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:00:56.821175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: warnings.warn(msg) Aug 01 12:00:56.821175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: Aug 01 12:00:56.822393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:00:56.904064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '981f877ab34246c78b7dd24180359c78'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:00:56.929966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '981f877ab34246c78b7dd24180359c78'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:00:56.945944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '981f877ab34246c78b7dd24180359c78'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:00:56.946786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '981f877ab34246c78b7dd24180359c78'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:00:56.947573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '981f877ab34246c78b7dd24180359c78'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:00:56.952171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:00:56.965707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 752/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:00:56 2022] DELETE /identity/v3/users/981f877ab34246c78b7dd24180359c78 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:00:56.971301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff8ada-77cf-4a95-8a83-01239a3c9fa7 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:56.971752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff8ada-77cf-4a95-8a83-01239a3c9fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:56.972086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff8ada-77cf-4a95-8a83-01239a3c9fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:56.995019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d3ff8ada-77cf-4a95-8a83-01239a3c9fa7 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:00:56.996269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d3ff8ada-77cf-4a95-8a83-01239a3c9fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:00:57.007179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:00:57.076581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 756/1508] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:00:57.088196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:00:57.089863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:00:57.183098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05b58b-b1d2-4c1b-9b83-4214e9211427 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:00:57.187100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e085b399-632e-455a-87d0-9529b251f9f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:00:57.187750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085b399-632e-455a-87d0-9529b251f9f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:57.188051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085b399-632e-455a-87d0-9529b251f9f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:57.188317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085b399-632e-455a-87d0-9529b251f9f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:57.199309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e085b399-632e-455a-87d0-9529b251f9f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:00:57.201059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e085b399-632e-455a-87d0-9529b251f9f3 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:00:57.212212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 753/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:00:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:00:57.598174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff8ada-77cf-4a95-8a83-01239a3c9fa7 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:00:57.599758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d3ff8ada-77cf-4a95-8a83-01239a3c9fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:00:57.613002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff8ada-77cf-4a95-8a83-01239a3c9fa7 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:00:57.624649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:00:57.625392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:00:57.625602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:00:57.625870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] PATH_INFO: `/v3/projects/5a68455333574c789b72235d31b7fef2` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:00:57.627219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a68455333574c789b72235d31b7fef2)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:00:57.628274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WARNING py.warnings [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:00:57.628274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: warnings.warn(msg) Aug 01 12:00:57.628274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: Aug 01 12:00:57.629531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:00:57.701475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a68455333574c789b72235d31b7fef2', 'request_id': 'req-3e39af89-7dba-4033-aae2-088640008371'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:00:57.715360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a68455333574c789b72235d31b7fef2', 'request_id': 'req-3e39af89-7dba-4033-aae2-088640008371'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:00:57.728032 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a68455333574c789b72235d31b7fef2', 'request_id': 'req-3e39af89-7dba-4033-aae2-088640008371'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:00:57.742510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] The token cache is being invalidate because project 5a68455333574c789b72235d31b7fef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:00:57.742800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:00:57.743610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3e39af89-7dba-4033-aae2-088640008371 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:00:57.745206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 757/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:00:57 2022] DELETE /identity/v3/projects/5a68455333574c789b72235d31b7fef2 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:01:07.207449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:01:07.738963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:02:39.390498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4f4bc-bebc-49b9-a9b8-2eeb1180ebff None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:39.391166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4f4bc-bebc-49b9-a9b8-2eeb1180ebff None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:39.391166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4f4bc-bebc-49b9-a9b8-2eeb1180ebff None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:39.392615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 12:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:02:39.398777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a68aa8-fbf6-4c01-8f90-0880946ead52 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:39.399352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a68aa8-fbf6-4c01-8f90-0880946ead52 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:39.399352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a68aa8-fbf6-4c01-8f90-0880946ead52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:39.425800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-58a68aa8-fbf6-4c01-8f90-0880946ead52 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:02:39.427067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-58a68aa8-fbf6-4c01-8f90-0880946ead52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:39.493210 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 12:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:02:39.662979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4f4bc-bebc-49b9-a9b8-2eeb1180ebff None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:02:39.666124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-11f4f4bc-bebc-49b9-a9b8-2eeb1180ebff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:39.793788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4f4bc-bebc-49b9-a9b8-2eeb1180ebff None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:02:39.800424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580520ea-52d7-4b0c-ab6f-39bf48f29bc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:02:39.801081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580520ea-52d7-4b0c-ab6f-39bf48f29bc5 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:39.801362 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580520ea-52d7-4b0c-ab6f-39bf48f29bc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:39.801635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580520ea-52d7-4b0c-ab6f-39bf48f29bc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:39.815688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580520ea-52d7-4b0c-ab6f-39bf48f29bc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:02:39.817365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580520ea-52d7-4b0c-ab6f-39bf48f29bc5 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:02:39.829757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 755/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:02:40.747485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecae648-052c-4bc9-b7f4-91c423309a36 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:40.748379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecae648-052c-4bc9-b7f4-91c423309a36 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:40.748379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecae648-052c-4bc9-b7f4-91c423309a36 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:40.749538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 12:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:02:40.756701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74e4b1-db73-4c9a-98a7-689e165b10d6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:40.757325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74e4b1-db73-4c9a-98a7-689e165b10d6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:40.757325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74e4b1-db73-4c9a-98a7-689e165b10d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:40.781180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-7d74e4b1-db73-4c9a-98a7-689e165b10d6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:02:40.782455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7d74e4b1-db73-4c9a-98a7-689e165b10d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:40.793427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 12:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:02:40.957495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecae648-052c-4bc9-b7f4-91c423309a36 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:02:40.976559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecae648-052c-4bc9-b7f4-91c423309a36 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:02:40.984024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cfae5-bcd3-44c3-a12b-84fe8334b45a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:02:40.984875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cfae5-bcd3-44c3-a12b-84fe8334b45a service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:40.985215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cfae5-bcd3-44c3-a12b-84fe8334b45a service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:40.985513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cfae5-bcd3-44c3-a12b-84fe8334b45a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:40.999668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cfae5-bcd3-44c3-a12b-84fe8334b45a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:02:41.002353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cfae5-bcd3-44c3-a12b-84fe8334b45a service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:02:41.017243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 760/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:02:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:02:41.459610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d49e36-7d4f-404c-8f8a-27270dd6e776 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:41.459888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d49e36-7d4f-404c-8f8a-27270dd6e776 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:41.460198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d49e36-7d4f-404c-8f8a-27270dd6e776 None None] PATH_INFO: `/v3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:41.461562 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 757/1517] 10.222.0.17 () {54 vars in 925 bytes} [Mon Aug 1 12:02:41 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:02:41.468654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b979d-c6a8-404f-b569-44d45863dcd5 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:41.468936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b979d-c6a8-404f-b569-44d45863dcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:41.469260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b979d-c6a8-404f-b569-44d45863dcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:41.533986 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8c4b979d-c6a8-404f-b569-44d45863dcd5 None None] MFA Rules not processed for user `9db663d35f964d42938be0d1a9ed4071`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:02:41.536149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8c4b979d-c6a8-404f-b569-44d45863dcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:41.629461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 761/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Aug 1 12:02:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:02:45.499551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44143539-208d-4eef-b41a-5ac9129ec2b6 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:45.500158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44143539-208d-4eef-b41a-5ac9129ec2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:45.500158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44143539-208d-4eef-b41a-5ac9129ec2b6 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:45.501469 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 758/1519] 10.222.0.49 () {54 vars in 961 bytes} [Mon Aug 1 12:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 12:02:45.507421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a81ab-ab54-47d5-9f6a-aa5a3231a03a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:45.508036 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a81ab-ab54-47d5-9f6a-aa5a3231a03a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:45.508036 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a81ab-ab54-47d5-9f6a-aa5a3231a03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:45.530267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-320a81ab-ab54-47d5-9f6a-aa5a3231a03a None None] MFA Rules not processed for user `9db663d35f964d42938be0d1a9ed4071`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:02:45.531755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-320a81ab-ab54-47d5-9f6a-aa5a3231a03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:45.541249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 762/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Aug 1 12:02:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:02:45.549832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44143539-208d-4eef-b41a-5ac9129ec2b6 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:02:45.564565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44143539-208d-4eef-b41a-5ac9129ec2b6 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:02:45.570108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b84e9-393b-4e34-a3cc-f7607ee43438 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:02:45.570832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b84e9-393b-4e34-a3cc-f7607ee43438 service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:45.571166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b84e9-393b-4e34-a3cc-f7607ee43438 service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:45.571544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b84e9-393b-4e34-a3cc-f7607ee43438 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:45.582229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b84e9-393b-4e34-a3cc-f7607ee43438 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:02:45.584555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b84e9-393b-4e34-a3cc-f7607ee43438 service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:02:45.599435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 759/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:02:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:02:52.296381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ab0bc-fc6d-4fc5-b906-96b12b22f0ed None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:52.296934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ab0bc-fc6d-4fc5-b906-96b12b22f0ed None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:52.296934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ab0bc-fc6d-4fc5-b906-96b12b22f0ed None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:52.298188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 12:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 12:02:52.303825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71790b-1917-4418-a892-75b13bcb5937 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:52.304392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71790b-1917-4418-a892-75b13bcb5937 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:52.304392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71790b-1917-4418-a892-75b13bcb5937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:52.325776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-6f71790b-1917-4418-a892-75b13bcb5937 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:02:52.327023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6f71790b-1917-4418-a892-75b13bcb5937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:52.336520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 12:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:02:52.496476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ab0bc-fc6d-4fc5-b906-96b12b22f0ed None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:02:52.513202 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ab0bc-fc6d-4fc5-b906-96b12b22f0ed None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:02:52.519216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf3be2-d1c0-42ad-9992-2c3ebc116fcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:02:52.519843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf3be2-d1c0-42ad-9992-2c3ebc116fcd service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:52.520122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf3be2-d1c0-42ad-9992-2c3ebc116fcd service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:52.520436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf3be2-d1c0-42ad-9992-2c3ebc116fcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:52.529574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf3be2-d1c0-42ad-9992-2c3ebc116fcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:02:52.531231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf3be2-d1c0-42ad-9992-2c3ebc116fcd service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:02:52.541576 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 764/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:02:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:02:53.404970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88dcb-b4a9-4a6b-b7f3-4b79b26a10ee None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:53.405546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88dcb-b4a9-4a6b-b7f3-4b79b26a10ee None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:53.405546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88dcb-b4a9-4a6b-b7f3-4b79b26a10ee None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:53.407489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 1 12:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:02:53.412797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99aef-bfd1-4ce3-82d8-7651f57790f2 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:53.413301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99aef-bfd1-4ce3-82d8-7651f57790f2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:53.413406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99aef-bfd1-4ce3-82d8-7651f57790f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:53.433874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-27d99aef-bfd1-4ce3-82d8-7651f57790f2 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:02:53.435106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-27d99aef-bfd1-4ce3-82d8-7651f57790f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:53.443455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 1 12:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:02:53.600004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88dcb-b4a9-4a6b-b7f3-4b79b26a10ee None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:02:53.615660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88dcb-b4a9-4a6b-b7f3-4b79b26a10ee None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:02:53.621936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b750d1-7ce0-4046-b1ab-1d2d474d362f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:02:53.622592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b750d1-7ce0-4046-b1ab-1d2d474d362f service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:53.622875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b750d1-7ce0-4046-b1ab-1d2d474d362f service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:53.623301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b750d1-7ce0-4046-b1ab-1d2d474d362f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:53.632681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b750d1-7ce0-4046-b1ab-1d2d474d362f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:02:53.634301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b750d1-7ce0-4046-b1ab-1d2d474d362f service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:02:53.645435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 762/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:02:57.580121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e4d9-5c4a-440c-8685-87a937eb3bc5 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:57.581054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e4d9-5c4a-440c-8685-87a937eb3bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:57.581231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e4d9-5c4a-440c-8685-87a937eb3bc5 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:57.582826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 12:02:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:02:57.590454 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014fe04-fc06-4604-9f29-ad5e00500018 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:57.591061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014fe04-fc06-4604-9f29-ad5e00500018 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:57.591243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014fe04-fc06-4604-9f29-ad5e00500018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:57.622321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e014fe04-fc06-4604-9f29-ad5e00500018 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:02:57.624544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e014fe04-fc06-4604-9f29-ad5e00500018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:57.638617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 12:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:02:57.722067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8e4d9-5c4a-440c-8685-87a937eb3bc5 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:02:57.724097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6ce8e4d9-5c4a-440c-8685-87a937eb3bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:57.757378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:02:57.790937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8e4d9-5c4a-440c-8685-87a937eb3bc5 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:02:57.794747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff18aa1-5c7c-420b-840b-5ff1282f6017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:02:57.795532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff18aa1-5c7c-420b-840b-5ff1282f6017 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:57.795809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff18aa1-5c7c-420b-840b-5ff1282f6017 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:57.796082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff18aa1-5c7c-420b-840b-5ff1282f6017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:57.806845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff18aa1-5c7c-420b-840b-5ff1282f6017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:02:57.808387 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff18aa1-5c7c-420b-840b-5ff1282f6017 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:02:57.819614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 767/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:02:59.858309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff5e64-fd22-4bcd-9b11-73f4de96d06d None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:59.858932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff5e64-fd22-4bcd-9b11-73f4de96d06d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:59.858932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff5e64-fd22-4bcd-9b11-73f4de96d06d None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:59.860440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 12:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:02:59.866921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db2a65-be36-4edf-acfe-3407f21a7d29 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:59.868147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db2a65-be36-4edf-acfe-3407f21a7d29 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:59.868147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db2a65-be36-4edf-acfe-3407f21a7d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:59.889399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d4db2a65-be36-4edf-acfe-3407f21a7d29 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:02:59.890624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d4db2a65-be36-4edf-acfe-3407f21a7d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:59.898780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 12:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:02:59.936768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff5e64-fd22-4bcd-9b11-73f4de96d06d None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:02:59.938634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6fff5e64-fd22-4bcd-9b11-73f4de96d06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:02:59.956520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff5e64-fd22-4bcd-9b11-73f4de96d06d None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:02:59.962480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f889385c-b2df-4048-a035-26ef06b4d8e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:02:59.963317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889385c-b2df-4048-a035-26ef06b4d8e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:02:59.963813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889385c-b2df-4048-a035-26ef06b4d8e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:02:59.964321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f889385c-b2df-4048-a035-26ef06b4d8e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:02:59.981238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889385c-b2df-4048-a035-26ef06b4d8e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:02:59.984093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f889385c-b2df-4048-a035-26ef06b4d8e2 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:03:00.002919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 765/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:03:03.723464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da651761-4da4-4103-9bc2-8e90f52d01f3 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:03:03.724015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da651761-4da4-4103-9bc2-8e90f52d01f3 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:03:03.724015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da651761-4da4-4103-9bc2-8e90f52d01f3 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:03:03.725441 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 12:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:03:03.730814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7d782-0514-4e8e-a1d2-0e30d4db16ec None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:03:03.731319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7d782-0514-4e8e-a1d2-0e30d4db16ec None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:03:03.731319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7d782-0514-4e8e-a1d2-0e30d4db16ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:03:03.752650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-fcf7d782-0514-4e8e-a1d2-0e30d4db16ec None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:03:03.753889 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fcf7d782-0514-4e8e-a1d2-0e30d4db16ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:03:03.762180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 12:03:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:03:04.078196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da651761-4da4-4103-9bc2-8e90f52d01f3 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:03:04.080307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-da651761-4da4-4103-9bc2-8e90f52d01f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:03:04.140580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da651761-4da4-4103-9bc2-8e90f52d01f3 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:03:04.144418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ddc0e-40aa-47e2-aad5-d7423abb8482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:03:04.145060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ddc0e-40aa-47e2-aad5-d7423abb8482 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:03:04.145338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ddc0e-40aa-47e2-aad5-d7423abb8482 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:03:04.145627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ddc0e-40aa-47e2-aad5-d7423abb8482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:03:04.155639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ddc0e-40aa-47e2-aad5-d7423abb8482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:03:04.157210 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ddc0e-40aa-47e2-aad5-d7423abb8482 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:03:04.169312 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 770/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:03:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:03:06.209947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7c3be-ea23-401c-9f3a-c87f4f14be07 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:03:06.210620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7c3be-ea23-401c-9f3a-c87f4f14be07 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:03:06.210620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7c3be-ea23-401c-9f3a-c87f4f14be07 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:03:06.212389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Aug 1 12:03:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:03:06.219963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc67848-7718-4834-ad5f-c6b5de155262 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:03:06.220502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc67848-7718-4834-ad5f-c6b5de155262 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:03:06.220502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc67848-7718-4834-ad5f-c6b5de155262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:03:06.245905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3dc67848-7718-4834-ad5f-c6b5de155262 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:03:06.247117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3dc67848-7718-4834-ad5f-c6b5de155262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:03:06.257174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Aug 1 12:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:03:06.575089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7c3be-ea23-401c-9f3a-c87f4f14be07 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:03:06.577973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-4ea7c3be-ea23-401c-9f3a-c87f4f14be07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:03:06.597727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7c3be-ea23-401c-9f3a-c87f4f14be07 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:03:06.601752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfa7ed-cd1a-400b-a6cc-de417269217d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:03:06.602407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfa7ed-cd1a-400b-a6cc-de417269217d service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:03:06.602684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfa7ed-cd1a-400b-a6cc-de417269217d service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:03:06.602962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfa7ed-cd1a-400b-a6cc-de417269217d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:03:06.612404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfa7ed-cd1a-400b-a6cc-de417269217d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:03:06.613846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfa7ed-cd1a-400b-a6cc-de417269217d service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:03:06.623907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 768/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:03:16.239496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=47 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:03:16.619684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=82 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:37.983022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54ad56-58a2-42c0-9cec-0e5e700248bb None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:37.983707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54ad56-58a2-42c0-9cec-0e5e700248bb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:37.983707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54ad56-58a2-42c0-9cec-0e5e700248bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.015988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-ae54ad56-58a2-42c0-9cec-0e5e700248bb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.017271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ae54ad56-58a2-42c0-9cec-0e5e700248bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.027210 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 772/1540] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 01 12:04:38.034684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a1ed2-0924-4d29-8aba-e2c8b2bf4c61 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.035466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a1ed2-0924-4d29-8aba-e2c8b2bf4c61 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.035654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505a1ed2-0924-4d29-8aba-e2c8b2bf4c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.063011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694a963-80a6-4ffc-b7e5-a99be033c91e None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.063403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694a963-80a6-4ffc-b7e5-a99be033c91e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.063575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3694a963-80a6-4ffc-b7e5-a99be033c91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.086023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-505a1ed2-0924-4d29-8aba-e2c8b2bf4c61 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.088706 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-505a1ed2-0924-4d29-8aba-e2c8b2bf4c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.092186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3694a963-80a6-4ffc-b7e5-a99be033c91e None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.093438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3694a963-80a6-4ffc-b7e5-a99be033c91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.109998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 773/1541] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:04:38.115283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694a963-80a6-4ffc-b7e5-a99be033c91e None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.122818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 769/1542] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.127006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3694a963-80a6-4ffc-b7e5-a99be033c91e None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.132947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cef2a-d6a4-4a4e-9b97-9d130843c9e6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.133456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cef2a-d6a4-4a4e-9b97-9d130843c9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.133945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cef2a-d6a4-4a4e-9b97-9d130843c9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.137565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ec8f6-d547-44da-82e3-0ff91d09bb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.138189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ec8f6-d547-44da-82e3-0ff91d09bb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.138474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ec8f6-d547-44da-82e3-0ff91d09bb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.138852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ec8f6-d547-44da-82e3-0ff91d09bb2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.139594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ec8f6-d547-44da-82e3-0ff91d09bb2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:04:38.139904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ec8f6-d547-44da-82e3-0ff91d09bb2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.141595 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ec8f6-d547-44da-82e3-0ff91d09bb2b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.161860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 774/1543] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:04:38.170405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ec8f6-d547-44da-82e3-0ff91d09bb2b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.187334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ec8f6-d547-44da-82e3-0ff91d09bb2b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.192477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f8a71-718c-4b8e-9016-8401d221d6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.193206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f8a71-718c-4b8e-9016-8401d221d6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.193488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f8a71-718c-4b8e-9016-8401d221d6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.193765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f8a71-718c-4b8e-9016-8401d221d6e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.194614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f8a71-718c-4b8e-9016-8401d221d6e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.195678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WARNING py.warnings [None req-903f8a71-718c-4b8e-9016-8401d221d6e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:04:38.195678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: warnings.warn(msg) Aug 01 12:04:38.195678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: Aug 01 12:04:38.197455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c23cef2a-d6a4-4a4e-9b97-9d130843c9e6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.199573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c23cef2a-d6a4-4a4e-9b97-9d130843c9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.202360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f8a71-718c-4b8e-9016-8401d221d6e3 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.239091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 770/1544] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:04:38.240634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017d531-3c54-4d1c-bb65-fbe47d7339cf None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.241013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017d531-3c54-4d1c-bb65-fbe47d7339cf None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.241355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017d531-3c54-4d1c-bb65-fbe47d7339cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.252251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 775/1545] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:04:38.253310 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f8a71-718c-4b8e-9016-8401d221d6e3 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.285508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1017d531-3c54-4d1c-bb65-fbe47d7339cf None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.286851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1017d531-3c54-4d1c-bb65-fbe47d7339cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.376919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 771/1546] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.377715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017d531-3c54-4d1c-bb65-fbe47d7339cf None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.416322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017d531-3c54-4d1c-bb65-fbe47d7339cf None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.421145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f3826-462f-401d-8b79-390a2a5180ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.422299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f3826-462f-401d-8b79-390a2a5180ae admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.422612 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f3826-462f-401d-8b79-390a2a5180ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.422835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f3826-462f-401d-8b79-390a2a5180ae admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.424272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f3826-462f-401d-8b79-390a2a5180ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.428349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f8a71-718c-4b8e-9016-8401d221d6e3 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.428594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: WARNING py.warnings [None req-978f3826-462f-401d-8b79-390a2a5180ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:04:38.428594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: warnings.warn(msg) Aug 01 12:04:38.428594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: Aug 01 12:04:38.430515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f3826-462f-401d-8b79-390a2a5180ae admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.438196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a0890-7801-48b8-8f05-c56d0ff98e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.438843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a0890-7801-48b8-8f05-c56d0ff98e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.439121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a0890-7801-48b8-8f05-c56d0ff98e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.439389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a0890-7801-48b8-8f05-c56d0ff98e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.440102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a0890-7801-48b8-8f05-c56d0ff98e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:04:38.440435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a0890-7801-48b8-8f05-c56d0ff98e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.446800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a0890-7801-48b8-8f05-c56d0ff98e29 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.489762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 776/1547] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:04:38.491300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f94a0-e60c-4844-a31f-0c12330cd6eb None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.491584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f94a0-e60c-4844-a31f-0c12330cd6eb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.493830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 772/1548] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:38.495481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49571d-70a6-47b4-b1cc-823be264a656 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.495772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49571d-70a6-47b4-b1cc-823be264a656 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.495998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f94a0-e60c-4844-a31f-0c12330cd6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.498162 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49571d-70a6-47b4-b1cc-823be264a656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.534303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9f5f94a0-e60c-4844-a31f-0c12330cd6eb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.535669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ce49571d-70a6-47b4-b1cc-823be264a656 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.536165 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9f5f94a0-e60c-4844-a31f-0c12330cd6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.537097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ce49571d-70a6-47b4-b1cc-823be264a656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.545574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 777/1549] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.547167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f40aa1-4803-43b7-8fd9-6485d2f1ea90 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.547667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f40aa1-4803-43b7-8fd9-6485d2f1ea90 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.548160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f40aa1-4803-43b7-8fd9-6485d2f1ea90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.555168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 773/1550] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.556330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49571d-70a6-47b4-b1cc-823be264a656 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.567888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49571d-70a6-47b4-b1cc-823be264a656 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.572815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faeb0f3-931d-4242-90b5-025150754f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.573465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeb0f3-931d-4242-90b5-025150754f19 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.573747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeb0f3-931d-4242-90b5-025150754f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.574021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faeb0f3-931d-4242-90b5-025150754f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.575069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faeb0f3-931d-4242-90b5-025150754f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.575783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a6f40aa1-4803-43b7-8fd9-6485d2f1ea90 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.577169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faeb0f3-931d-4242-90b5-025150754f19 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.577617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a6f40aa1-4803-43b7-8fd9-6485d2f1ea90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.586569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 778/1551] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.587764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f40aa1-4803-43b7-8fd9-6485d2f1ea90 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.595633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 774/1552] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:38.599218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680769a-b4f6-44c0-a288-dd2b7b633030 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.599218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680769a-b4f6-44c0-a288-dd2b7b633030 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.599218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680769a-b4f6-44c0-a288-dd2b7b633030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.603714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f40aa1-4803-43b7-8fd9-6485d2f1ea90 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.609046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477345e7-aeab-460c-b5eb-4e44cacb7d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.609806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477345e7-aeab-460c-b5eb-4e44cacb7d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.610188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477345e7-aeab-460c-b5eb-4e44cacb7d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.610541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477345e7-aeab-460c-b5eb-4e44cacb7d95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.611293 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477345e7-aeab-460c-b5eb-4e44cacb7d95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.613599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477345e7-aeab-460c-b5eb-4e44cacb7d95 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.625034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3680769a-b4f6-44c0-a288-dd2b7b633030 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.626278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3680769a-b4f6-44c0-a288-dd2b7b633030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.635360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 775/1553] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.636460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ebe53-dbc5-479f-aeb0-19f7a3c23b18 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.636750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ebe53-dbc5-479f-aeb0-19f7a3c23b18 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.637017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ebe53-dbc5-479f-aeb0-19f7a3c23b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.649097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 779/1554] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/projects => generated 403 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:38.649955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477345e7-aeab-460c-b5eb-4e44cacb7d95 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.661098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-797ebe53-dbc5-479f-aeb0-19f7a3c23b18 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.662325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-797ebe53-dbc5-479f-aeb0-19f7a3c23b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.730404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477345e7-aeab-460c-b5eb-4e44cacb7d95 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.735388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04446267-0aea-4de1-929f-867bffb78b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.736049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04446267-0aea-4de1-929f-867bffb78b82 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.736347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04446267-0aea-4de1-929f-867bffb78b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.736626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04446267-0aea-4de1-929f-867bffb78b82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.737336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04446267-0aea-4de1-929f-867bffb78b82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:04:38.737635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04446267-0aea-4de1-929f-867bffb78b82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.739384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04446267-0aea-4de1-929f-867bffb78b82 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.743079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 776/1555] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.745085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775afa84-9f8c-4d19-a758-ca0ec23cc99f None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.745343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775afa84-9f8c-4d19-a758-ca0ec23cc99f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.745618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775afa84-9f8c-4d19-a758-ca0ec23cc99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.761975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 780/1556] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:04:38.761975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04446267-0aea-4de1-929f-867bffb78b82 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.761975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:38.763867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04446267-0aea-4de1-929f-867bffb78b82 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.772140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b52e3-10a1-4247-8ccd-e1663b521a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.772837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b52e3-10a1-4247-8ccd-e1663b521a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.773119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b52e3-10a1-4247-8ccd-e1663b521a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.773361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b52e3-10a1-4247-8ccd-e1663b521a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.774108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b52e3-10a1-4247-8ccd-e1663b521a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.775703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934b52e3-10a1-4247-8ccd-e1663b521a7e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.779908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-775afa84-9f8c-4d19-a758-ca0ec23cc99f None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.781179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-775afa84-9f8c-4d19-a758-ca0ec23cc99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.787135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 781/1557] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:38.788526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e2568-bf0e-49fe-8614-a5a186c7ba99 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.788817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e2568-bf0e-49fe-8614-a5a186c7ba99 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.789091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e2568-bf0e-49fe-8614-a5a186c7ba99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.793418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 777/1558] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.793418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c59386-f30b-4a2d-a7be-998ce2ce1a19 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.793418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c59386-f30b-4a2d-a7be-998ce2ce1a19 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.793418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c59386-f30b-4a2d-a7be-998ce2ce1a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.810533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d1c59386-f30b-4a2d-a7be-998ce2ce1a19 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.811016 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-426e2568-bf0e-49fe-8614-a5a186c7ba99 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.811738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d1c59386-f30b-4a2d-a7be-998ce2ce1a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.812268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-426e2568-bf0e-49fe-8614-a5a186c7ba99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.819649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 778/1559] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.820823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 782/1560] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.821521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e2568-bf0e-49fe-8614-a5a186c7ba99 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.825909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fefdd-b63c-49e5-83b5-c7a83b4917b8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.826666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fefdd-b63c-49e5-83b5-c7a83b4917b8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.827143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fefdd-b63c-49e5-83b5-c7a83b4917b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.835533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e2568-bf0e-49fe-8614-a5a186c7ba99 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.840738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebcc91-8938-4d69-b134-0113030fdf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.841353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebcc91-8938-4d69-b134-0113030fdf3b admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.841636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebcc91-8938-4d69-b134-0113030fdf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.841944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebcc91-8938-4d69-b134-0113030fdf3b admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.842581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebcc91-8938-4d69-b134-0113030fdf3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.844574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebcc91-8938-4d69-b134-0113030fdf3b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.846973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-319fefdd-b63c-49e5-83b5-c7a83b4917b8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.848178 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-319fefdd-b63c-49e5-83b5-c7a83b4917b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.856237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 779/1561] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:38.857129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fefdd-b63c-49e5-83b5-c7a83b4917b8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.866959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fefdd-b63c-49e5-83b5-c7a83b4917b8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.871829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e2b66-3a5f-4919-a0ef-c741643928f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.872006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 783/1562] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/users => generated 302 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:38.875307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e2b66-3a5f-4919-a0ef-c741643928f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.875307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e2b66-3a5f-4919-a0ef-c741643928f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.875547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebcc91-8938-4d69-b134-0113030fdf3b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.875653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e2b66-3a5f-4919-a0ef-c741643928f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.875653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e2b66-3a5f-4919-a0ef-c741643928f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:04:38.875653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e2b66-3a5f-4919-a0ef-c741643928f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.875653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: WARNING py.warnings [None req-268e2b66-3a5f-4919-a0ef-c741643928f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:04:38.875653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: warnings.warn(msg) Aug 01 12:04:38.875653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: Aug 01 12:04:38.876156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e2b66-3a5f-4919-a0ef-c741643928f5 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.884828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebcc91-8938-4d69-b134-0113030fdf3b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.888361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 780/1563] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:04:38.889215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e2b66-3a5f-4919-a0ef-c741643928f5 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.889897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c02e9-8671-4460-8288-4dce557eee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.890730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c02e9-8671-4460-8288-4dce557eee4d admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.891177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c02e9-8671-4460-8288-4dce557eee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.891617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c02e9-8671-4460-8288-4dce557eee4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.892466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c02e9-8671-4460-8288-4dce557eee4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.894562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c02e9-8671-4460-8288-4dce557eee4d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.900517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e2b66-3a5f-4919-a0ef-c741643928f5 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:38.905017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb609539-4567-4a12-87e1-872c8331ee7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:38.905677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb609539-4567-4a12-87e1-872c8331ee7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.905954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb609539-4567-4a12-87e1-872c8331ee7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.906228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb609539-4567-4a12-87e1-872c8331ee7f admin admin] PATH_INFO: `/v3/projects/5fd75f5d11444549bfa2d273599bb098/users/443696f19aec46fc9cd59bd9d61587f5/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.918958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:38.923255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb609539-4567-4a12-87e1-872c8331ee7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd75f5d11444549bfa2d273599bb098, user_id=443696f19aec46fc9cd59bd9d61587f5, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:38.923510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 784/1564] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:38.924371 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c02e9-8671-4460-8288-4dce557eee4d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:38.926597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb609539-4567-4a12-87e1-872c8331ee7f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:38.941196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 781/1565] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:38 2022] PUT /identity/v3/projects/5fd75f5d11444549bfa2d273599bb098/users/443696f19aec46fc9cd59bd9d61587f5/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:38.945171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ad0fe-8b28-4e3a-86e2-bc63da3e7b37 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:38.945171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ad0fe-8b28-4e3a-86e2-bc63da3e7b37 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:38.945171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ad0fe-8b28-4e3a-86e2-bc63da3e7b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:38.963435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d55ad0fe-8b28-4e3a-86e2-bc63da3e7b37 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:38.964695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d55ad0fe-8b28-4e3a-86e2-bc63da3e7b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:38.990466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c02e9-8671-4460-8288-4dce557eee4d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.041229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 782/1566] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:39.042583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df165e-03bd-46d5-82c3-d4a132080a9b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.042865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df165e-03bd-46d5-82c3-d4a132080a9b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.043137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8df165e-03bd-46d5-82c3-d4a132080a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.062394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e8df165e-03bd-46d5-82c3-d4a132080a9b None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:39.063622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e8df165e-03bd-46d5-82c3-d4a132080a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:39.066028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce4fb5-21f5-466a-9f99-c0ef6c7cc369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.066676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce4fb5-21f5-466a-9f99-c0ef6c7cc369 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.066958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce4fb5-21f5-466a-9f99-c0ef6c7cc369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.067231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce4fb5-21f5-466a-9f99-c0ef6c7cc369 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.068087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce4fb5-21f5-466a-9f99-c0ef6c7cc369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:04:39.068429 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce4fb5-21f5-466a-9f99-c0ef6c7cc369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.070066 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce4fb5-21f5-466a-9f99-c0ef6c7cc369 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.072960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 783/1567] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:39.073869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df165e-03bd-46d5-82c3-d4a132080a9b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.082977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 785/1568] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:04:39.083880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce4fb5-21f5-466a-9f99-c0ef6c7cc369 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.084125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8df165e-03bd-46d5-82c3-d4a132080a9b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.089805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c190a-674f-42b5-90c4-5571a604cfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.090629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c190a-674f-42b5-90c4-5571a604cfbf admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.090987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c190a-674f-42b5-90c4-5571a604cfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.091326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c190a-674f-42b5-90c4-5571a604cfbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.092208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c190a-674f-42b5-90c4-5571a604cfbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:04:39.092541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c190a-674f-42b5-90c4-5571a604cfbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.094184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c190a-674f-42b5-90c4-5571a604cfbf admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.095758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce4fb5-21f5-466a-9f99-c0ef6c7cc369 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.101768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f279d93-2a1f-4388-ab54-be054bf96aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.102529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f279d93-2a1f-4388-ab54-be054bf96aee admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.102896 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f279d93-2a1f-4388-ab54-be054bf96aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.103265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f279d93-2a1f-4388-ab54-be054bf96aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.104112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f279d93-2a1f-4388-ab54-be054bf96aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.106203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f279d93-2a1f-4388-ab54-be054bf96aee admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.115201 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 784/1569] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:04:39.116427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c190a-674f-42b5-90c4-5571a604cfbf admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.121334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 786/1570] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:39.122180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f279d93-2a1f-4388-ab54-be054bf96aee admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.129403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c190a-674f-42b5-90c4-5571a604cfbf admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.134862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3ae51-11ae-4a9d-a8b3-236010907620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.135061 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f279d93-2a1f-4388-ab54-be054bf96aee admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.135633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3ae51-11ae-4a9d-a8b3-236010907620 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.135984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3ae51-11ae-4a9d-a8b3-236010907620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.136357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3ae51-11ae-4a9d-a8b3-236010907620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.137124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3ae51-11ae-4a9d-a8b3-236010907620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.139130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3ae51-11ae-4a9d-a8b3-236010907620 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.139627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c63fe-e012-4096-9869-a353a7731742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.140245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c63fe-e012-4096-9869-a353a7731742 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.140557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c63fe-e012-4096-9869-a353a7731742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.140837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c63fe-e012-4096-9869-a353a7731742 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.141501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c63fe-e012-4096-9869-a353a7731742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.143414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c63fe-e012-4096-9869-a353a7731742 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.167859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 787/1571] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:39.170206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 785/1572] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:39.171257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3ae51-11ae-4a9d-a8b3-236010907620 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.172457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb644be2-178a-4a24-a81d-df0aeca66c53 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.172905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb644be2-178a-4a24-a81d-df0aeca66c53 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.173278 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb644be2-178a-4a24-a81d-df0aeca66c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.197633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-fb644be2-178a-4a24-a81d-df0aeca66c53 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:39.198925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-fb644be2-178a-4a24-a81d-df0aeca66c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:39.214832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:39.246600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3ae51-11ae-4a9d-a8b3-236010907620 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.252746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355389e-1ab5-4e02-98cc-7f048f166e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.253385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355389e-1ab5-4e02-98cc-7f048f166e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.253667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355389e-1ab5-4e02-98cc-7f048f166e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.253939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355389e-1ab5-4e02-98cc-7f048f166e77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.254655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355389e-1ab5-4e02-98cc-7f048f166e77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:04:39.255014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355389e-1ab5-4e02-98cc-7f048f166e77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.257004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8355389e-1ab5-4e02-98cc-7f048f166e77 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.268554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 788/1573] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:39.276979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb644be2-178a-4a24-a81d-df0aeca66c53 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.277754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 786/1574] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:04:39.277754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355389e-1ab5-4e02-98cc-7f048f166e77 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.281080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355389e-1ab5-4e02-98cc-7f048f166e77 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.281990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb644be2-178a-4a24-a81d-df0aeca66c53 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.286136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da602fd-b5ae-4b8f-a3f5-9b0543a145b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.286821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da602fd-b5ae-4b8f-a3f5-9b0543a145b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.287152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4d4aa-27b1-43c5-99b1-26bf8ea6dca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.287268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da602fd-b5ae-4b8f-a3f5-9b0543a145b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.287498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da602fd-b5ae-4b8f-a3f5-9b0543a145b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.287952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4d4aa-27b1-43c5-99b1-26bf8ea6dca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.288193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da602fd-b5ae-4b8f-a3f5-9b0543a145b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.288461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4d4aa-27b1-43c5-99b1-26bf8ea6dca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.288743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4d4aa-27b1-43c5-99b1-26bf8ea6dca0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.289448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4d4aa-27b1-43c5-99b1-26bf8ea6dca0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:04:39.289764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4d4aa-27b1-43c5-99b1-26bf8ea6dca0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.290150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da602fd-b5ae-4b8f-a3f5-9b0543a145b1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.291366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4d4aa-27b1-43c5-99b1-26bf8ea6dca0 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.304161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 789/1575] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:04:39.305075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4d4aa-27b1-43c5-99b1-26bf8ea6dca0 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.317006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4d4aa-27b1-43c5-99b1-26bf8ea6dca0 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.319163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 787/1576] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:39.320077 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da602fd-b5ae-4b8f-a3f5-9b0543a145b1 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.344873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:39.380624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6ace6-e763-4c8e-9f90-37f41f041074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.381257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6ace6-e763-4c8e-9f90-37f41f041074 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.381535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6ace6-e763-4c8e-9f90-37f41f041074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.381804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6ace6-e763-4c8e-9f90-37f41f041074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.382465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6ace6-e763-4c8e-9f90-37f41f041074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.384405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6ace6-e763-4c8e-9f90-37f41f041074 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.387561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da602fd-b5ae-4b8f-a3f5-9b0543a145b1 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.392663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7e466-c890-4f1e-9c12-f87bd62ee043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.393236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e466-c890-4f1e-9c12-f87bd62ee043 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.393516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e466-c890-4f1e-9c12-f87bd62ee043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.393793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7e466-c890-4f1e-9c12-f87bd62ee043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.394494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7e466-c890-4f1e-9c12-f87bd62ee043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.396172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7e466-c890-4f1e-9c12-f87bd62ee043 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.406679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 788/1577] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:39.407593 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7e466-c890-4f1e-9c12-f87bd62ee043 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.413775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 790/1578] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:39.414572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6ace6-e763-4c8e-9f90-37f41f041074 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.423393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7e466-c890-4f1e-9c12-f87bd62ee043 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.481132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6ace6-e763-4c8e-9f90-37f41f041074 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.486791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280acbdc-0615-48b4-b35e-4dfe96c83615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.487076 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3f945-c080-492c-9a60-3d3c612a6528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.487523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280acbdc-0615-48b4-b35e-4dfe96c83615 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.487897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280acbdc-0615-48b4-b35e-4dfe96c83615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.488206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3f945-c080-492c-9a60-3d3c612a6528 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.488378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280acbdc-0615-48b4-b35e-4dfe96c83615 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.488489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3f945-c080-492c-9a60-3d3c612a6528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.488832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3f945-c080-492c-9a60-3d3c612a6528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.488969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280acbdc-0615-48b4-b35e-4dfe96c83615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.489735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3f945-c080-492c-9a60-3d3c612a6528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.490976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280acbdc-0615-48b4-b35e-4dfe96c83615 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.491448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3f945-c080-492c-9a60-3d3c612a6528 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.502585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 789/1579] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:39.503753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3f945-c080-492c-9a60-3d3c612a6528 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.515156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3f945-c080-492c-9a60-3d3c612a6528 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.518569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 791/1580] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/users => generated 302 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:39.519636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280acbdc-0615-48b4-b35e-4dfe96c83615 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.519991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4c1e7-24d1-42f5-85c3-41f3ed6a5426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.520673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4c1e7-24d1-42f5-85c3-41f3ed6a5426 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.520988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4c1e7-24d1-42f5-85c3-41f3ed6a5426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.521263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4c1e7-24d1-42f5-85c3-41f3ed6a5426 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.521941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4c1e7-24d1-42f5-85c3-41f3ed6a5426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.524572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4c1e7-24d1-42f5-85c3-41f3ed6a5426 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.547962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:39.558541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 790/1581] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/projects => generated 389 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:39.559439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4c1e7-24d1-42f5-85c3-41f3ed6a5426 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.584272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280acbdc-0615-48b4-b35e-4dfe96c83615 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.646967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4c1e7-24d1-42f5-85c3-41f3ed6a5426 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.647622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57aae8-cf10-4ed7-94fd-981fb19f8af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.648278 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57aae8-cf10-4ed7-94fd-981fb19f8af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.648692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57aae8-cf10-4ed7-94fd-981fb19f8af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.649094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57aae8-cf10-4ed7-94fd-981fb19f8af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.653100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d59b4-5c27-4e82-b384-acf8435198c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.653767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d59b4-5c27-4e82-b384-acf8435198c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.654041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d59b4-5c27-4e82-b384-acf8435198c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.654307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d59b4-5c27-4e82-b384-acf8435198c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.654987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d59b4-5c27-4e82-b384-acf8435198c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.657177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d59b4-5c27-4e82-b384-acf8435198c8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.662102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57aae8-cf10-4ed7-94fd-981fb19f8af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.664930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57aae8-cf10-4ed7-94fd-981fb19f8af1 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.683845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 792/1582] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:39.685421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57aae8-cf10-4ed7-94fd-981fb19f8af1 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.690481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 791/1583] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:39.690481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d59b4-5c27-4e82-b384-acf8435198c8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.703150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57aae8-cf10-4ed7-94fd-981fb19f8af1 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.721704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:39.778055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25388a4-f24f-4e69-9e30-f2c11c49c324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.778869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25388a4-f24f-4e69-9e30-f2c11c49c324 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.779156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25388a4-f24f-4e69-9e30-f2c11c49c324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.779433 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25388a4-f24f-4e69-9e30-f2c11c49c324 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.780251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25388a4-f24f-4e69-9e30-f2c11c49c324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.782797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25388a4-f24f-4e69-9e30-f2c11c49c324 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.790297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d59b4-5c27-4e82-b384-acf8435198c8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.797628 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f86795-9490-44c6-9a85-3e8f95d80e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.798612 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f86795-9490-44c6-9a85-3e8f95d80e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.798947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f86795-9490-44c6-9a85-3e8f95d80e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.799313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f86795-9490-44c6-9a85-3e8f95d80e69 admin admin] PATH_INFO: `/v3/projects/279d5a3294934195bb335826a36aaf62/users/01390dc3eacb4c08bde3e97540d0d1c3/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.813378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 793/1584] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:39.813904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25388a4-f24f-4e69-9e30-f2c11c49c324 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.821403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f86795-9490-44c6-9a85-3e8f95d80e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=279d5a3294934195bb335826a36aaf62, user_id=01390dc3eacb4c08bde3e97540d0d1c3, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.824845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f86795-9490-44c6-9a85-3e8f95d80e69 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.829656 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25388a4-f24f-4e69-9e30-f2c11c49c324 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.835666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07738b71-6f22-4120-be3e-555e40b43895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.836377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07738b71-6f22-4120-be3e-555e40b43895 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.836668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07738b71-6f22-4120-be3e-555e40b43895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.836947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07738b71-6f22-4120-be3e-555e40b43895 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.837747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07738b71-6f22-4120-be3e-555e40b43895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.840232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07738b71-6f22-4120-be3e-555e40b43895 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.856021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 792/1585] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:39 2022] PUT /identity/v3/projects/279d5a3294934195bb335826a36aaf62/users/01390dc3eacb4c08bde3e97540d0d1c3/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:39.857823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f86795-9490-44c6-9a85-3e8f95d80e69 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.876865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 794/1586] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/users => generated 296 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:39.878017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07738b71-6f22-4120-be3e-555e40b43895 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.890455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07738b71-6f22-4120-be3e-555e40b43895 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.942470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f86795-9490-44c6-9a85-3e8f95d80e69 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:39.949826 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a4aa3-e650-496d-b887-48692183adaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.950695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a4aa3-e650-496d-b887-48692183adaa admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.951150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a4aa3-e650-496d-b887-48692183adaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.951579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a4aa3-e650-496d-b887-48692183adaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.952515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a4aa3-e650-496d-b887-48692183adaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.955319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361a4aa3-e650-496d-b887-48692183adaa admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:39.969248 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:39.979752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 793/1587] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:39.981006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a4aa3-e650-496d-b887-48692183adaa admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:39.987002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5a796-4bae-42fc-9dfb-7389a666726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:39.988224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5a796-4bae-42fc-9dfb-7389a666726d admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:39.988488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5a796-4bae-42fc-9dfb-7389a666726d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:39.988811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5a796-4bae-42fc-9dfb-7389a666726d admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:39.990056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5a796-4bae-42fc-9dfb-7389a666726d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:39.992043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5a796-4bae-42fc-9dfb-7389a666726d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.005351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361a4aa3-e650-496d-b887-48692183adaa admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.011920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5962e9-61ad-4a70-bc5c-1bab4524a020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.013152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 795/1588] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:40.013558 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5962e9-61ad-4a70-bc5c-1bab4524a020 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.014108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5962e9-61ad-4a70-bc5c-1bab4524a020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.014798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5962e9-61ad-4a70-bc5c-1bab4524a020 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.015102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5a796-4bae-42fc-9dfb-7389a666726d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.015940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5962e9-61ad-4a70-bc5c-1bab4524a020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.019188 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5962e9-61ad-4a70-bc5c-1bab4524a020 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.036083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5a796-4bae-42fc-9dfb-7389a666726d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.043843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f50e66-3f8b-41be-98c6-ec436dc2a18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.045010 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f50e66-3f8b-41be-98c6-ec436dc2a18c admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.045552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f50e66-3f8b-41be-98c6-ec436dc2a18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.046055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f50e66-3f8b-41be-98c6-ec436dc2a18c admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.047085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f50e66-3f8b-41be-98c6-ec436dc2a18c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.048238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 794/1589] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:39 2022] POST /identity/v3/users => generated 295 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:40.049271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5962e9-61ad-4a70-bc5c-1bab4524a020 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.054765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f50e66-3f8b-41be-98c6-ec436dc2a18c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.061355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5962e9-61ad-4a70-bc5c-1bab4524a020 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.068231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd639aea-e3a7-4ed5-bf04-4c4a1869c168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.069194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd639aea-e3a7-4ed5-bf04-4c4a1869c168 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.069653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd639aea-e3a7-4ed5-bf04-4c4a1869c168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.070212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd639aea-e3a7-4ed5-bf04-4c4a1869c168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.071267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd639aea-e3a7-4ed5-bf04-4c4a1869c168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.073353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd639aea-e3a7-4ed5-bf04-4c4a1869c168 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.085698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 796/1590] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:40 2022] POST /identity/v3/users => generated 286 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:40.088156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb46381-c5b9-488a-a8ec-b14dd36c898a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.089580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb46381-c5b9-488a-a8ec-b14dd36c898a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.089580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb46381-c5b9-488a-a8ec-b14dd36c898a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.094535 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 795/1591] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:40.095549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd639aea-e3a7-4ed5-bf04-4c4a1869c168 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.110203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd639aea-e3a7-4ed5-bf04-4c4a1869c168 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.115678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606329d4-4667-4194-a08f-0a6954f67670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.116438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606329d4-4667-4194-a08f-0a6954f67670 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.116795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606329d4-4667-4194-a08f-0a6954f67670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.118704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606329d4-4667-4194-a08f-0a6954f67670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.118907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3bb46381-c5b9-488a-a8ec-b14dd36c898a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:40.119088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606329d4-4667-4194-a08f-0a6954f67670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.119810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3bb46381-c5b9-488a-a8ec-b14dd36c898a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:40.120840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606329d4-4667-4194-a08f-0a6954f67670 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.136385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 797/1592] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:40.136385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb46381-c5b9-488a-a8ec-b14dd36c898a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.136859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 796/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:40.136859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606329d4-4667-4194-a08f-0a6954f67670 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.145337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb46381-c5b9-488a-a8ec-b14dd36c898a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.156190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266de30f-e5e3-40d0-8bc8-b059b2d8205d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.156190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266de30f-e5e3-40d0-8bc8-b059b2d8205d admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.156190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266de30f-e5e3-40d0-8bc8-b059b2d8205d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.156190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266de30f-e5e3-40d0-8bc8-b059b2d8205d admin admin] PATH_INFO: `/v3/projects/9126d4b7b81640b19ba5f62236e19444/users/9bc6d25504f1428a84ee2cd93657b9d1/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.163465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606329d4-4667-4194-a08f-0a6954f67670 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.168991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90e724-4825-416b-b29a-34766d796a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.170090 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90e724-4825-416b-b29a-34766d796a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.170090 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90e724-4825-416b-b29a-34766d796a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.170292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90e724-4825-416b-b29a-34766d796a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.170884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90e724-4825-416b-b29a-34766d796a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.172801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90e724-4825-416b-b29a-34766d796a6c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.177211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266de30f-e5e3-40d0-8bc8-b059b2d8205d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9126d4b7b81640b19ba5f62236e19444, user_id=9bc6d25504f1428a84ee2cd93657b9d1, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.178368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WARNING py.warnings [None req-266de30f-e5e3-40d0-8bc8-b059b2d8205d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:04:40.178368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: warnings.warn(msg) Aug 01 12:04:40.178368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: Aug 01 12:04:40.180745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266de30f-e5e3-40d0-8bc8-b059b2d8205d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.188759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 797/1594] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:40.190123 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90e724-4825-416b-b29a-34766d796a6c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.197758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 798/1595] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:40 2022] PUT /identity/v3/projects/9126d4b7b81640b19ba5f62236e19444/users/9bc6d25504f1428a84ee2cd93657b9d1/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:40.202655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266de30f-e5e3-40d0-8bc8-b059b2d8205d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.203637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90e724-4825-416b-b29a-34766d796a6c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.260503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:40.281430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5f406-69cc-4ff9-ac24-0c8cff996d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.282390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5f406-69cc-4ff9-ac24-0c8cff996d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.282843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5f406-69cc-4ff9-ac24-0c8cff996d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.283269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5f406-69cc-4ff9-ac24-0c8cff996d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.284482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5f406-69cc-4ff9-ac24-0c8cff996d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.287210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5f406-69cc-4ff9-ac24-0c8cff996d77 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.288954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266de30f-e5e3-40d0-8bc8-b059b2d8205d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.294692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a375d1d-9475-4ab8-9e3e-de7ace47e97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.295464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a375d1d-9475-4ab8-9e3e-de7ace47e97e admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.295804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a375d1d-9475-4ab8-9e3e-de7ace47e97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.296146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a375d1d-9475-4ab8-9e3e-de7ace47e97e admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.296967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a375d1d-9475-4ab8-9e3e-de7ace47e97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.298727 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a375d1d-9475-4ab8-9e3e-de7ace47e97e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.300004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 798/1596] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:40.304424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5f406-69cc-4ff9-ac24-0c8cff996d77 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.310960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 799/1597] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:40.311940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a375d1d-9475-4ab8-9e3e-de7ace47e97e admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.315734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5f406-69cc-4ff9-ac24-0c8cff996d77 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.321729 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2d92f-45fb-4b82-8609-430019ae50c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.322517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2d92f-45fb-4b82-8609-430019ae50c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.322903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2d92f-45fb-4b82-8609-430019ae50c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.323529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2d92f-45fb-4b82-8609-430019ae50c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.323654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a375d1d-9475-4ab8-9e3e-de7ace47e97e admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.324041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2d92f-45fb-4b82-8609-430019ae50c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.326076 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2d92f-45fb-4b82-8609-430019ae50c7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.328388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11717096-16cf-4620-93ee-76fb5d563d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.329418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11717096-16cf-4620-93ee-76fb5d563d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.329418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11717096-16cf-4620-93ee-76fb5d563d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.329699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11717096-16cf-4620-93ee-76fb5d563d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.330519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11717096-16cf-4620-93ee-76fb5d563d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.332248 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11717096-16cf-4620-93ee-76fb5d563d98 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.337439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 799/1598] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:40.338046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2d92f-45fb-4b82-8609-430019ae50c7 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.350804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 800/1599] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:40.352130 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11717096-16cf-4620-93ee-76fb5d563d98 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.364011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11717096-16cf-4620-93ee-76fb5d563d98 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.372875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1af657-cf36-4569-9166-696771d13438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.373720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1af657-cf36-4569-9166-696771d13438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.374300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1af657-cf36-4569-9166-696771d13438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.374445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1af657-cf36-4569-9166-696771d13438 admin admin] PATH_INFO: `/v3/projects/b75d70430685440e869b8b40c2b2c05d/users/f0332a9eefd64871b490c84123c4c9f3/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.395679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1af657-cf36-4569-9166-696771d13438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75d70430685440e869b8b40c2b2c05d, user_id=f0332a9eefd64871b490c84123c4c9f3, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.398875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1af657-cf36-4569-9166-696771d13438 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.406831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2d92f-45fb-4b82-8609-430019ae50c7 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.411964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322ffb0-701f-4c02-89f6-18fc0807e8c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.413118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ffb0-701f-4c02-89f6-18fc0807e8c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.413592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ffb0-701f-4c02-89f6-18fc0807e8c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.414064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ffb0-701f-4c02-89f6-18fc0807e8c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.418924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 801/1600] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:40 2022] PUT /identity/v3/projects/b75d70430685440e869b8b40c2b2c05d/users/f0332a9eefd64871b490c84123c4c9f3/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:40.420253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e5242-68e5-4431-95a9-c49835b638f5 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.420551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e5242-68e5-4431-95a9-c49835b638f5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.420821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e5242-68e5-4431-95a9-c49835b638f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.439818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322ffb0-701f-4c02-89f6-18fc0807e8c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.441872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322ffb0-701f-4c02-89f6-18fc0807e8c8 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.444251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8f1e5242-68e5-4431-95a9-c49835b638f5 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:40.445517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8f1e5242-68e5-4431-95a9-c49835b638f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:40.454870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 800/1601] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:40.455670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322ffb0-701f-4c02-89f6-18fc0807e8c8 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.466027 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322ffb0-701f-4c02-89f6-18fc0807e8c8 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.537242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc90c1-49c4-4135-a611-b6b089b0448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.537957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc90c1-49c4-4135-a611-b6b089b0448b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.538195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc90c1-49c4-4135-a611-b6b089b0448b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.538606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc90c1-49c4-4135-a611-b6b089b0448b admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.539326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc90c1-49c4-4135-a611-b6b089b0448b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.540342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 802/1602] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:40.540997 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc90c1-49c4-4135-a611-b6b089b0448b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.541377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e5242-68e5-4431-95a9-c49835b638f5 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.548867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:40.561450 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e5242-68e5-4431-95a9-c49835b638f5 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.561748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 801/1603] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:40.562684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc90c1-49c4-4135-a611-b6b089b0448b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.575139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551bd2c-98f9-4ea4-b7ab-06199cc02da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.576068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551bd2c-98f9-4ea4-b7ab-06199cc02da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.576542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551bd2c-98f9-4ea4-b7ab-06199cc02da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.577015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551bd2c-98f9-4ea4-b7ab-06199cc02da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.577717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc90c1-49c4-4135-a611-b6b089b0448b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.577871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551bd2c-98f9-4ea4-b7ab-06199cc02da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.579745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0551bd2c-98f9-4ea4-b7ab-06199cc02da3 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.585994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a1058-79c4-4e8e-b3ba-17749e8fc624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.587121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1058-79c4-4e8e-b3ba-17749e8fc624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.587732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1058-79c4-4e8e-b3ba-17749e8fc624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.588097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1058-79c4-4e8e-b3ba-17749e8fc624 admin admin] PATH_INFO: `/v3/projects/f10697a6130a4f2b9c0d4e5db1897d8a/users/e613776b2bf247c482774a55f97a3872/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.592942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 803/1604] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:40.593900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551bd2c-98f9-4ea4-b7ab-06199cc02da3 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.604981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551bd2c-98f9-4ea4-b7ab-06199cc02da3 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.609439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a1058-79c4-4e8e-b3ba-17749e8fc624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10697a6130a4f2b9c0d4e5db1897d8a, user_id=e613776b2bf247c482774a55f97a3872, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.610349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac36c89-cf09-482e-90af-8866da5ec2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.611161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac36c89-cf09-482e-90af-8866da5ec2e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.611542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac36c89-cf09-482e-90af-8866da5ec2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.611919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac36c89-cf09-482e-90af-8866da5ec2e5 admin admin] PATH_INFO: `/v3/projects/7dc50e555f674cc7b33ac8f28b6fbd33/users/fc8a5fa04b6141329778221169afdfc1/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.612244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a1058-79c4-4e8e-b3ba-17749e8fc624 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.630494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac36c89-cf09-482e-90af-8866da5ec2e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc50e555f674cc7b33ac8f28b6fbd33, user_id=fc8a5fa04b6141329778221169afdfc1, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.644390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 802/1605] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:40 2022] PUT /identity/v3/projects/f10697a6130a4f2b9c0d4e5db1897d8a/users/e613776b2bf247c482774a55f97a3872/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:40.644618 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac36c89-cf09-482e-90af-8866da5ec2e5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.644719 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6630ae1-b25b-4b25-ab99-9b5ac7eacc34 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.644719 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6630ae1-b25b-4b25-ab99-9b5ac7eacc34 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.644719 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6630ae1-b25b-4b25-ab99-9b5ac7eacc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.664455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 804/1606] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:40 2022] PUT /identity/v3/projects/7dc50e555f674cc7b33ac8f28b6fbd33/users/fc8a5fa04b6141329778221169afdfc1/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:40.664455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac36c89-cf09-482e-90af-8866da5ec2e5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.681835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-f6630ae1-b25b-4b25-ab99-9b5ac7eacc34 None None] MFA Rules not processed for user `f0332a9eefd64871b490c84123c4c9f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:40.683103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f6630ae1-b25b-4b25-ab99-9b5ac7eacc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:40.740993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:40.754274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac36c89-cf09-482e-90af-8866da5ec2e5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.760626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37744301-9f21-40b5-8b62-2f1a3452b72b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.761557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37744301-9f21-40b5-8b62-2f1a3452b72b service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.761937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37744301-9f21-40b5-8b62-2f1a3452b72b service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.762323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37744301-9f21-40b5-8b62-2f1a3452b72b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.804376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 803/1607] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:40.805330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6630ae1-b25b-4b25-ab99-9b5ac7eacc34 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.919010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6630ae1-b25b-4b25-ab99-9b5ac7eacc34 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.921520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37744301-9f21-40b5-8b62-2f1a3452b72b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.926187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf024e-8669-4fa2-b184-038a7dd3e2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.927230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf024e-8669-4fa2-b184-038a7dd3e2e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.928968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf024e-8669-4fa2-b184-038a7dd3e2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.929490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf024e-8669-4fa2-b184-038a7dd3e2e3 admin admin] PATH_INFO: `/v3/projects/1971a84abd0145358138a0ef1b2501a9/users/e98a16b117294e10b9d0c8c721d8e114/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.930521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37744301-9f21-40b5-8b62-2f1a3452b72b service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.947092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 805/1608] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:40.948961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37744301-9f21-40b5-8b62-2f1a3452b72b service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:40.952872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf024e-8669-4fa2-b184-038a7dd3e2e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1971a84abd0145358138a0ef1b2501a9, user_id=e98a16b117294e10b9d0c8c721d8e114, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:40.957816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf024e-8669-4fa2-b184-038a7dd3e2e3 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:40.960828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37744301-9f21-40b5-8b62-2f1a3452b72b service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:40.968360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e3991-9d50-47ad-9fb1-fd22d1026584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:40.970028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e3991-9d50-47ad-9fb1-fd22d1026584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.974014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e3991-9d50-47ad-9fb1-fd22d1026584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.974401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e3991-9d50-47ad-9fb1-fd22d1026584 admin admin] PATH_INFO: `/v3/projects/3ec35db0316b42c4a54563701eb54c5b/users/4167faace34a428eb582380e4902bd3f/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:40.981993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 804/1609] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:40 2022] PUT /identity/v3/projects/1971a84abd0145358138a0ef1b2501a9/users/e98a16b117294e10b9d0c8c721d8e114/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:40.986351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de8e0f-eeac-4aac-a7ff-dbebc56b59f6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:40.989460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de8e0f-eeac-4aac-a7ff-dbebc56b59f6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:40.991597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de8e0f-eeac-4aac-a7ff-dbebc56b59f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:41.003929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e3991-9d50-47ad-9fb1-fd22d1026584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec35db0316b42c4a54563701eb54c5b, user_id=4167faace34a428eb582380e4902bd3f, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:41.007480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e3991-9d50-47ad-9fb1-fd22d1026584 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:41.038765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 806/1610] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:40 2022] PUT /identity/v3/projects/3ec35db0316b42c4a54563701eb54c5b/users/4167faace34a428eb582380e4902bd3f/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:41.038765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faabea-cbe5-401d-8978-ea933ac6f713 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:41.038765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faabea-cbe5-401d-8978-ea933ac6f713 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:41.038765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2faabea-cbe5-401d-8978-ea933ac6f713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:41.051895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b9de8e0f-eeac-4aac-a7ff-dbebc56b59f6 None None] MFA Rules not processed for user `e613776b2bf247c482774a55f97a3872`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:41.056636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b9de8e0f-eeac-4aac-a7ff-dbebc56b59f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:41.070873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-f2faabea-cbe5-401d-8978-ea933ac6f713 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:41.072575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f2faabea-cbe5-401d-8978-ea933ac6f713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:41.218509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:41.248438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 807/1611] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:41.249514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faabea-cbe5-401d-8978-ea933ac6f713 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:41.269016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 805/1612] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:41.270142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd955bf-7515-43b3-9ce8-09af65aa0d21 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:41.271421 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd955bf-7515-43b3-9ce8-09af65aa0d21 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:41.272166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd955bf-7515-43b3-9ce8-09af65aa0d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:41.273329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2faabea-cbe5-401d-8978-ea933ac6f713 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:41.283864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0738f-ba60-4f6c-a514-964cbe5d5320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:41.284551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0738f-ba60-4f6c-a514-964cbe5d5320 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:41.284832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0738f-ba60-4f6c-a514-964cbe5d5320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:41.285108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e0738f-ba60-4f6c-a514-964cbe5d5320 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:41.285789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0738f-ba60-4f6c-a514-964cbe5d5320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:41.287828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e0738f-ba60-4f6c-a514-964cbe5d5320 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:41.316698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-7cd955bf-7515-43b3-9ce8-09af65aa0d21 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:41.318072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7cd955bf-7515-43b3-9ce8-09af65aa0d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:41.343733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 806/1613] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:41.350627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a107e7-4912-4101-bc96-4f30749a5e8b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:41.350944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a107e7-4912-4101-bc96-4f30749a5e8b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:41.351674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a107e7-4912-4101-bc96-4f30749a5e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:41.356185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 808/1614] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:41 2022] POST /identity/v3/projects => generated 403 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:41.356998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0738f-ba60-4f6c-a514-964cbe5d5320 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:41.446544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-54a107e7-4912-4101-bc96-4f30749a5e8b None None] MFA Rules not processed for user `4167faace34a428eb582380e4902bd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:41.448612 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-54a107e7-4912-4101-bc96-4f30749a5e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:41.541999 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e0738f-ba60-4f6c-a514-964cbe5d5320 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:41.544820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:41.577895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6e4d1-89df-4940-b89b-6bb7071d6661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:41.578802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6e4d1-89df-4940-b89b-6bb7071d6661 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:41.579089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6e4d1-89df-4940-b89b-6bb7071d6661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:41.579365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6e4d1-89df-4940-b89b-6bb7071d6661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:41.580063 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6e4d1-89df-4940-b89b-6bb7071d6661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:41.582089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6e4d1-89df-4940-b89b-6bb7071d6661 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:41.644256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 807/1615] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:41.647901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 809/1616] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:41 2022] POST /identity/v3/projects => generated 379 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:41.647901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6e4d1-89df-4940-b89b-6bb7071d6661 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:41.649328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a107e7-4912-4101-bc96-4f30749a5e8b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:41.756772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a107e7-4912-4101-bc96-4f30749a5e8b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:41.771880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6e4d1-89df-4940-b89b-6bb7071d6661 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:41.779959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15ab9-6465-47a1-8b08-30e48fb4f13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:41.781013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15ab9-6465-47a1-8b08-30e48fb4f13c admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:41.781549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15ab9-6465-47a1-8b08-30e48fb4f13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:41.782097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb15ab9-6465-47a1-8b08-30e48fb4f13c admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:41.783231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb15ab9-6465-47a1-8b08-30e48fb4f13c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:41.785491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb15ab9-6465-47a1-8b08-30e48fb4f13c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:41.827335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 810/1617] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:41 2022] POST /identity/v3/users => generated 308 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:41.829323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15ab9-6465-47a1-8b08-30e48fb4f13c admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:41.841593 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8899e-7a6e-47f3-aedc-27c90013160b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:41.842264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8899e-7a6e-47f3-aedc-27c90013160b service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:41.842631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8899e-7a6e-47f3-aedc-27c90013160b service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:41.842953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8899e-7a6e-47f3-aedc-27c90013160b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:41.858863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8899e-7a6e-47f3-aedc-27c90013160b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:41.860506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8899e-7a6e-47f3-aedc-27c90013160b service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:41.876479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 808/1618] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:41.877422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8899e-7a6e-47f3-aedc-27c90013160b service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:41.893240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8899e-7a6e-47f3-aedc-27c90013160b service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:41.902609 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28b87a-c6d0-45aa-9d50-5e7470576725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:41.903249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28b87a-c6d0-45aa-9d50-5e7470576725 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:41.903617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28b87a-c6d0-45aa-9d50-5e7470576725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:41.903949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28b87a-c6d0-45aa-9d50-5e7470576725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:41.905219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28b87a-c6d0-45aa-9d50-5e7470576725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:41.908177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28b87a-c6d0-45aa-9d50-5e7470576725 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:41.912078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:41.929051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb15ab9-6465-47a1-8b08-30e48fb4f13c admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:41.945147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72a769-01ee-46f6-b0a7-60362e6a2c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:41.945840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72a769-01ee-46f6-b0a7-60362e6a2c76 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:41.946169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72a769-01ee-46f6-b0a7-60362e6a2c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:41.946620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72a769-01ee-46f6-b0a7-60362e6a2c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:41.962527 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 809/1619] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:41 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:41.970606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28b87a-c6d0-45aa-9d50-5e7470576725 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:41.984754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72a769-01ee-46f6-b0a7-60362e6a2c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:41.987619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72a769-01ee-46f6-b0a7-60362e6a2c76 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.018086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 811/1620] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:42.019838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656344-5d60-493c-96a6-a22774b090dc None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.020109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656344-5d60-493c-96a6-a22774b090dc None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.020568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f656344-5d60-493c-96a6-a22774b090dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.075794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2f656344-5d60-493c-96a6-a22774b090dc None None] MFA Rules not processed for user `01390dc3eacb4c08bde3e97540d0d1c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:42.077441 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2f656344-5d60-493c-96a6-a22774b090dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:42.098980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28b87a-c6d0-45aa-9d50-5e7470576725 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.109167 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4039c0-d148-4d85-8c14-a2c8602f8898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.110539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4039c0-d148-4d85-8c14-a2c8602f8898 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.111026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4039c0-d148-4d85-8c14-a2c8602f8898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.111382 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4039c0-d148-4d85-8c14-a2c8602f8898 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.112395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4039c0-d148-4d85-8c14-a2c8602f8898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.115641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4039c0-d148-4d85-8c14-a2c8602f8898 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.139712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:42.160026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 810/1621] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:41 2022] POST /identity/v3/users => generated 296 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:42.162607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4039c0-d148-4d85-8c14-a2c8602f8898 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.181689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4039c0-d148-4d85-8c14-a2c8602f8898 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.189758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7786b9-8816-4e4c-a750-8d27d1655f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.190979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7786b9-8816-4e4c-a750-8d27d1655f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.191350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7786b9-8816-4e4c-a750-8d27d1655f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.192237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7786b9-8816-4e4c-a750-8d27d1655f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.193514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7786b9-8816-4e4c-a750-8d27d1655f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.198495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7786b9-8816-4e4c-a750-8d27d1655f57 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.199683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 812/1622] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:42.200429 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656344-5d60-493c-96a6-a22774b090dc None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.219921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 811/1623] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:42.220106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f656344-5d60-493c-96a6-a22774b090dc None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.220745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7786b9-8816-4e4c-a750-8d27d1655f57 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.230423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4255a-d665-45fa-a081-dffdd52328e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.231315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4255a-d665-45fa-a081-dffdd52328e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.232196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4255a-d665-45fa-a081-dffdd52328e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.232540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4255a-d665-45fa-a081-dffdd52328e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.233237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4255a-d665-45fa-a081-dffdd52328e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.233984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7786b9-8816-4e4c-a750-8d27d1655f57 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.237024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4255a-d665-45fa-a081-dffdd52328e2 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.242013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ff145-ad17-4fd6-a3c7-d35bee578640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.242702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ff145-ad17-4fd6-a3c7-d35bee578640 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.242982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ff145-ad17-4fd6-a3c7-d35bee578640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.243708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ff145-ad17-4fd6-a3c7-d35bee578640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.244759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ff145-ad17-4fd6-a3c7-d35bee578640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.246450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ff145-ad17-4fd6-a3c7-d35bee578640 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.270500 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 812/1624] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:42.280895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ff145-ad17-4fd6-a3c7-d35bee578640 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.284216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 813/1625] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:42 2022] POST /identity/v3/users => generated 293 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:42.287374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4255a-d665-45fa-a081-dffdd52328e2 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.301025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4255a-d665-45fa-a081-dffdd52328e2 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.303629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ff145-ad17-4fd6-a3c7-d35bee578640 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.314430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be1266-e4a7-492c-9d5e-81b5ea25d912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.326336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0afe8-f01d-453f-a654-4363921587db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.326906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1266-e4a7-492c-9d5e-81b5ea25d912 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.327305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1266-e4a7-492c-9d5e-81b5ea25d912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.327424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0afe8-f01d-453f-a654-4363921587db admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.327710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be1266-e4a7-492c-9d5e-81b5ea25d912 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.328087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0afe8-f01d-453f-a654-4363921587db admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.328583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be1266-e4a7-492c-9d5e-81b5ea25d912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.328947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0afe8-f01d-453f-a654-4363921587db admin admin] PATH_INFO: `/v3/projects/6bffc7ee232a4a0babd0e9b6fd649624/users/381fc3b652044bf5b8943ffcf4a00849/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.330868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be1266-e4a7-492c-9d5e-81b5ea25d912 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.353647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0afe8-f01d-453f-a654-4363921587db admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bffc7ee232a4a0babd0e9b6fd649624, user_id=381fc3b652044bf5b8943ffcf4a00849, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.357664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0afe8-f01d-453f-a654-4363921587db admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.385113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 814/1626] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:42 2022] PUT /identity/v3/projects/6bffc7ee232a4a0babd0e9b6fd649624/users/381fc3b652044bf5b8943ffcf4a00849/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:42.386843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0afe8-f01d-453f-a654-4363921587db admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.398250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 813/1627] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:42 2022] POST /identity/v3/projects => generated 403 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:42.399627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be1266-e4a7-492c-9d5e-81b5ea25d912 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.479092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0afe8-f01d-453f-a654-4363921587db admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.504824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be1266-e4a7-492c-9d5e-81b5ea25d912 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.510979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710eacd-abcf-4f62-9422-2a4b011ab933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.512901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710eacd-abcf-4f62-9422-2a4b011ab933 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.513376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710eacd-abcf-4f62-9422-2a4b011ab933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.514689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710eacd-abcf-4f62-9422-2a4b011ab933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.516386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710eacd-abcf-4f62-9422-2a4b011ab933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.518121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b710eacd-abcf-4f62-9422-2a4b011ab933 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.535184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 814/1628] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:42.540048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710eacd-abcf-4f62-9422-2a4b011ab933 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.543030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:42.555194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710eacd-abcf-4f62-9422-2a4b011ab933 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.563751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fd79b-3d79-4067-ac97-15e43a3d5a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.564223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd868021-663f-4d7e-b455-77638553515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.565113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fd79b-3d79-4067-ac97-15e43a3d5a5b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.565254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd868021-663f-4d7e-b455-77638553515a admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.565513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd868021-663f-4d7e-b455-77638553515a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.565955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fd79b-3d79-4067-ac97-15e43a3d5a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.566309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd868021-663f-4d7e-b455-77638553515a admin admin] PATH_INFO: `/v3/projects/eac9fa5c4c564bbfa256b9f28da9ee52/users/5e39fd4906a645c490f320ea3c09664f/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.566680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fd79b-3d79-4067-ac97-15e43a3d5a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.568003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fd79b-3d79-4067-ac97-15e43a3d5a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.570792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fd79b-3d79-4067-ac97-15e43a3d5a5b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.587412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 815/1629] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:42.587412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fd79b-3d79-4067-ac97-15e43a3d5a5b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.593995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd868021-663f-4d7e-b455-77638553515a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac9fa5c4c564bbfa256b9f28da9ee52, user_id=5e39fd4906a645c490f320ea3c09664f, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.602774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd868021-663f-4d7e-b455-77638553515a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.614532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fd79b-3d79-4067-ac97-15e43a3d5a5b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.626035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9103803-12bc-4394-845e-24662e3488fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.627146 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9103803-12bc-4394-845e-24662e3488fb admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.627529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9103803-12bc-4394-845e-24662e3488fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.628023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9103803-12bc-4394-845e-24662e3488fb admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.628957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9103803-12bc-4394-845e-24662e3488fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.629292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 815/1630] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:42 2022] PUT /identity/v3/projects/eac9fa5c4c564bbfa256b9f28da9ee52/users/5e39fd4906a645c490f320ea3c09664f/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:42.630353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd868021-663f-4d7e-b455-77638553515a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.632465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9103803-12bc-4394-845e-24662e3488fb admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.763440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 816/1631] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:42 2022] POST /identity/v3/users => generated 308 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:42.763440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9103803-12bc-4394-845e-24662e3488fb admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.763440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9103803-12bc-4394-845e-24662e3488fb admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.839222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd868021-663f-4d7e-b455-77638553515a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:42.853280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb4548-a50c-4b6d-910d-e22abd255451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.855316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb4548-a50c-4b6d-910d-e22abd255451 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.855316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb4548-a50c-4b6d-910d-e22abd255451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.855316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb4548-a50c-4b6d-910d-e22abd255451 admin admin] PATH_INFO: `/v3/projects/ea390cf3f402462e832de08237f16bc5/users/8822ef4610924beb90693e7e55412be9/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.879132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76e38a-010d-40d1-8da1-d208a9e82840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:42.880305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76e38a-010d-40d1-8da1-d208a9e82840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:42.880305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76e38a-010d-40d1-8da1-d208a9e82840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:42.880677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76e38a-010d-40d1-8da1-d208a9e82840 admin admin] PATH_INFO: `/v3/projects/6bffc7ee232a4a0babd0e9b6fd649624/users/381fc3b652044bf5b8943ffcf4a00849/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:42.882876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76e38a-010d-40d1-8da1-d208a9e82840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bffc7ee232a4a0babd0e9b6fd649624, user_id=381fc3b652044bf5b8943ffcf4a00849, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.886865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76e38a-010d-40d1-8da1-d208a9e82840 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.892118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb4548-a50c-4b6d-910d-e22abd255451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea390cf3f402462e832de08237f16bc5, user_id=8822ef4610924beb90693e7e55412be9, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:42.896319 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb4548-a50c-4b6d-910d-e22abd255451 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:42.901960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 817/1632] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:42 2022] PUT /identity/v3/projects/6bffc7ee232a4a0babd0e9b6fd649624/users/381fc3b652044bf5b8943ffcf4a00849/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:42.904164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76e38a-010d-40d1-8da1-d208a9e82840 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.933595 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 816/1633] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:42 2022] PUT /identity/v3/projects/ea390cf3f402462e832de08237f16bc5/users/8822ef4610924beb90693e7e55412be9/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:42.933595 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb4548-a50c-4b6d-910d-e22abd255451 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:42.934196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:43.019234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76e38a-010d-40d1-8da1-d208a9e82840 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:43.065657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb4548-a50c-4b6d-910d-e22abd255451 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:43.073424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c8f60-e1a7-4550-9ea0-eeeded9f8ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:43.074085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c8f60-e1a7-4550-9ea0-eeeded9f8ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:43.074363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c8f60-e1a7-4550-9ea0-eeeded9f8ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:43.074634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c8f60-e1a7-4550-9ea0-eeeded9f8ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:43.075917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c8f60-e1a7-4550-9ea0-eeeded9f8ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:43.077861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c8f60-e1a7-4550-9ea0-eeeded9f8ec2 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:43.095403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 817/1634] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:43.095403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20df6e-4042-4bfe-878e-57da03d8aab1 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:43.095403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20df6e-4042-4bfe-878e-57da03d8aab1 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:43.095817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20df6e-4042-4bfe-878e-57da03d8aab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:43.110210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfd8b9-eea3-48da-b1dc-112319815a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:43.111251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfd8b9-eea3-48da-b1dc-112319815a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:43.111706 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfd8b9-eea3-48da-b1dc-112319815a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:43.112282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcfd8b9-eea3-48da-b1dc-112319815a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:43.113173 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfd8b9-eea3-48da-b1dc-112319815a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:43.115259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcfd8b9-eea3-48da-b1dc-112319815a8e admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:43.133949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 818/1635] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:43.135181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfd8b9-eea3-48da-b1dc-112319815a8e admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:43.150718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcfd8b9-eea3-48da-b1dc-112319815a8e admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:43.159753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4fca7-5cf4-4c72-b447-4fcdc39f07a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:43.160632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4fca7-5cf4-4c72-b447-4fcdc39f07a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:43.161014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4fca7-5cf4-4c72-b447-4fcdc39f07a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:43.161511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4fca7-5cf4-4c72-b447-4fcdc39f07a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:43.162506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4fca7-5cf4-4c72-b447-4fcdc39f07a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:43.165610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4fca7-5cf4-4c72-b447-4fcdc39f07a1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:43.179406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-db20df6e-4042-4bfe-878e-57da03d8aab1 None None] MFA Rules not processed for user `381fc3b652044bf5b8943ffcf4a00849`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:43.185889 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-db20df6e-4042-4bfe-878e-57da03d8aab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:43.188926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 819/1636] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:43.189986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4fca7-5cf4-4c72-b447-4fcdc39f07a1 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:43.215469 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4fca7-5cf4-4c72-b447-4fcdc39f07a1 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:43.228959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480f1d8-4129-430b-b80c-ce603b15670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:43.229953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480f1d8-4129-430b-b80c-ce603b15670d admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:43.230333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480f1d8-4129-430b-b80c-ce603b15670d admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:43.230708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480f1d8-4129-430b-b80c-ce603b15670d admin admin] PATH_INFO: `/v3/projects/738dc937c3df435b8a23a53bf4dfea77/users/801ae1e600cb4aa1b84ee9cbc7cf0efb/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:43.272654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:43.274762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480f1d8-4129-430b-b80c-ce603b15670d admin admin] RBAC: Authorizing `identity:create_grant(project_id=738dc937c3df435b8a23a53bf4dfea77, user_id=801ae1e600cb4aa1b84ee9cbc7cf0efb, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:43.286616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480f1d8-4129-430b-b80c-ce603b15670d admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:43.328514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 820/1637] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:43 2022] PUT /identity/v3/projects/738dc937c3df435b8a23a53bf4dfea77/users/801ae1e600cb4aa1b84ee9cbc7cf0efb/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:43.331407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480f1d8-4129-430b-b80c-ce603b15670d admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:43.410061 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 818/1638] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:43 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:43.411690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20df6e-4042-4bfe-878e-57da03d8aab1 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:43.471644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20df6e-4042-4bfe-878e-57da03d8aab1 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:43.552153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480f1d8-4129-430b-b80c-ce603b15670d admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:43.562355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a18245-8c74-44d8-9d23-48454d232734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:43.563672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a18245-8c74-44d8-9d23-48454d232734 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:43.564599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a18245-8c74-44d8-9d23-48454d232734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:43.566983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a18245-8c74-44d8-9d23-48454d232734 admin admin] PATH_INFO: `/v3/projects/eac9fa5c4c564bbfa256b9f28da9ee52/users/5e39fd4906a645c490f320ea3c09664f/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:43.576201 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a18245-8c74-44d8-9d23-48454d232734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eac9fa5c4c564bbfa256b9f28da9ee52, user_id=5e39fd4906a645c490f320ea3c09664f, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:43.589955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a18245-8c74-44d8-9d23-48454d232734 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:43.596681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e42920-e926-4737-aaa9-84ed543634c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:43.597401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e42920-e926-4737-aaa9-84ed543634c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:43.597685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e42920-e926-4737-aaa9-84ed543634c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:43.598352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e42920-e926-4737-aaa9-84ed543634c7 admin admin] PATH_INFO: `/v3/projects/ea390cf3f402462e832de08237f16bc5/users/8822ef4610924beb90693e7e55412be9/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:43.600779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e42920-e926-4737-aaa9-84ed543634c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea390cf3f402462e832de08237f16bc5, user_id=8822ef4610924beb90693e7e55412be9, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:43.603822 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e42920-e926-4737-aaa9-84ed543634c7 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:43.610869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 821/1639] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:43 2022] PUT /identity/v3/projects/eac9fa5c4c564bbfa256b9f28da9ee52/users/5e39fd4906a645c490f320ea3c09664f/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:43.623440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a18245-8c74-44d8-9d23-48454d232734 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:43.625163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 819/1640] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:43 2022] PUT /identity/v3/projects/ea390cf3f402462e832de08237f16bc5/users/8822ef4610924beb90693e7e55412be9/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:43.632953 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e42920-e926-4737-aaa9-84ed543634c7 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:43.704190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e42920-e926-4737-aaa9-84ed543634c7 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:43.709903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d95538-8c92-4cc3-96b1-d7050c3892c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:43.710995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d95538-8c92-4cc3-96b1-d7050c3892c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:43.711540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d95538-8c92-4cc3-96b1-d7050c3892c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:43.712121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d95538-8c92-4cc3-96b1-d7050c3892c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:43.732579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:43.798417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d95538-8c92-4cc3-96b1-d7050c3892c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:43.800539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d95538-8c92-4cc3-96b1-d7050c3892c3 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:43.815031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 820/1641] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:43 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:43.817460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea4eb3-8142-4a1b-a95c-9d1acb750a46 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:43.818319 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea4eb3-8142-4a1b-a95c-9d1acb750a46 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:43.819106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea4eb3-8142-4a1b-a95c-9d1acb750a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:43.906333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9aea4eb3-8142-4a1b-a95c-9d1acb750a46 None None] MFA Rules not processed for user `5e39fd4906a645c490f320ea3c09664f`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:43.908219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9aea4eb3-8142-4a1b-a95c-9d1acb750a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:43.931848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:43.987037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a18245-8c74-44d8-9d23-48454d232734 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:44.106789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 821/1642] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:44.107814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5bb9a-0e4f-4ae7-91bb-cb4ac7eff3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:44.109493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f93d0-d08f-41fb-b54c-ca0b5fd112db None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:44.109936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f93d0-d08f-41fb-b54c-ca0b5fd112db None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:44.110357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f93d0-d08f-41fb-b54c-ca0b5fd112db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:44.110982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5bb9a-0e4f-4ae7-91bb-cb4ac7eff3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:44.111432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5bb9a-0e4f-4ae7-91bb-cb4ac7eff3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:44.111596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5bb9a-0e4f-4ae7-91bb-cb4ac7eff3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:44.112937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5bb9a-0e4f-4ae7-91bb-cb4ac7eff3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:44.118807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5bb9a-0e4f-4ae7-91bb-cb4ac7eff3c2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:44.144745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 822/1643] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 533 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:44.146070 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5bb9a-0e4f-4ae7-91bb-cb4ac7eff3c2 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:44.174348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5bb9a-0e4f-4ae7-91bb-cb4ac7eff3c2 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:44.185982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368626b-dd1b-482a-8f84-84303f0931ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:44.187079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368626b-dd1b-482a-8f84-84303f0931ca service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:44.187468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368626b-dd1b-482a-8f84-84303f0931ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:44.188101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e368626b-dd1b-482a-8f84-84303f0931ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:44.210827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368626b-dd1b-482a-8f84-84303f0931ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:44.219472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e368626b-dd1b-482a-8f84-84303f0931ca service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:44.227945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a88f93d0-d08f-41fb-b54c-ca0b5fd112db None None] MFA Rules not processed for user `8822ef4610924beb90693e7e55412be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:44.238797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a88f93d0-d08f-41fb-b54c-ca0b5fd112db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:44.241431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 823/1644] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:44.241975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368626b-dd1b-482a-8f84-84303f0931ca service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:44.274964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e368626b-dd1b-482a-8f84-84303f0931ca service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:44.295147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decedc2e-478d-4fd5-81d3-79ace934cab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:44.295922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decedc2e-478d-4fd5-81d3-79ace934cab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:44.296247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decedc2e-478d-4fd5-81d3-79ace934cab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:44.296692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decedc2e-478d-4fd5-81d3-79ace934cab8 admin admin] PATH_INFO: `/v3/projects/738dc937c3df435b8a23a53bf4dfea77/users/801ae1e600cb4aa1b84ee9cbc7cf0efb/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:44.309915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decedc2e-478d-4fd5-81d3-79ace934cab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738dc937c3df435b8a23a53bf4dfea77, user_id=801ae1e600cb4aa1b84ee9cbc7cf0efb, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:44.312742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decedc2e-478d-4fd5-81d3-79ace934cab8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:44.353498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 824/1645] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:44 2022] PUT /identity/v3/projects/738dc937c3df435b8a23a53bf4dfea77/users/801ae1e600cb4aa1b84ee9cbc7cf0efb/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:44.354504 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decedc2e-478d-4fd5-81d3-79ace934cab8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:44.427760 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 822/1646] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:44.429537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f93d0-d08f-41fb-b54c-ca0b5fd112db None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:44.447416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f93d0-d08f-41fb-b54c-ca0b5fd112db None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:44.504193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decedc2e-478d-4fd5-81d3-79ace934cab8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:44.512862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:44.514096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:44.514573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:44.515036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] PATH_INFO: `/v3/users/f0332a9eefd64871b490c84123c4c9f3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:44.517257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0332a9eefd64871b490c84123c4c9f3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:44.521952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:44.532916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191a1c1-702e-4962-83dd-9e14670c55c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:44.534663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191a1c1-702e-4962-83dd-9e14670c55c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:44.535161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191a1c1-702e-4962-83dd-9e14670c55c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:44.536257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191a1c1-702e-4962-83dd-9e14670c55c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:44.594750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:44.609480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:44.635845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191a1c1-702e-4962-83dd-9e14670c55c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:44.636984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0332a9eefd64871b490c84123c4c9f3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:44.638310 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191a1c1-702e-4962-83dd-9e14670c55c1 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:44.674107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0332a9eefd64871b490c84123c4c9f3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:44.674701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 823/1647] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:44.676263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191a1c1-702e-4962-83dd-9e14670c55c1 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:44.699638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0332a9eefd64871b490c84123c4c9f3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:44.700867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191a1c1-702e-4962-83dd-9e14670c55c1 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:44.701505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0332a9eefd64871b490c84123c4c9f3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:44.703138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d5350596-4913-4e4e-a304-0475893ce624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0332a9eefd64871b490c84123c4c9f3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:44.711804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de1d38-48ac-4ea8-b3ca-52567012bbe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:44.712484 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de1d38-48ac-4ea8-b3ca-52567012bbe7 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:44.713223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de1d38-48ac-4ea8-b3ca-52567012bbe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:44.713505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de1d38-48ac-4ea8-b3ca-52567012bbe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:44.719771 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-65de1d38-48ac-4ea8-b3ca-52567012bbe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:44.747920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 825/1648] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:04:44 2022] DELETE /identity/v3/users/f0332a9eefd64871b490c84123c4c9f3 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:44.750455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c61b5-fe98-4f19-8dcd-856c9bedc688 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:44.751158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c61b5-fe98-4f19-8dcd-856c9bedc688 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:44.751685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c61b5-fe98-4f19-8dcd-856c9bedc688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:44.859808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b39c61b5-fe98-4f19-8dcd-856c9bedc688 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:44.862232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b39c61b5-fe98-4f19-8dcd-856c9bedc688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:44.911465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de1d38-48ac-4ea8-b3ca-52567012bbe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:44.913045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de1d38-48ac-4ea8-b3ca-52567012bbe7 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:44.937169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 824/1649] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:45.041890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 826/1650] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:45.095112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de1d38-48ac-4ea8-b3ca-52567012bbe7 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:45.096768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-65de1d38-48ac-4ea8-b3ca-52567012bbe7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:45.110494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c61b5-fe98-4f19-8dcd-856c9bedc688 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:45.112125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b39c61b5-fe98-4f19-8dcd-856c9bedc688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:45.112761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de1d38-48ac-4ea8-b3ca-52567012bbe7 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:45.253318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dde144-f39c-4e4a-8d37-5416578d0aca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:45.253787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dde144-f39c-4e4a-8d37-5416578d0aca service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:45.254073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dde144-f39c-4e4a-8d37-5416578d0aca service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:45.254410 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dde144-f39c-4e4a-8d37-5416578d0aca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:45.257439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c6dde144-f39c-4e4a-8d37-5416578d0aca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:45.262079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c61b5-fe98-4f19-8dcd-856c9bedc688 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:45.266225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010f87-795f-4a3e-afaf-f77fc86890a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:45.266994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010f87-795f-4a3e-afaf-f77fc86890a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:45.267279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010f87-795f-4a3e-afaf-f77fc86890a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:45.268003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a010f87-795f-4a3e-afaf-f77fc86890a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:45.280050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010f87-795f-4a3e-afaf-f77fc86890a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:45.281973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a010f87-795f-4a3e-afaf-f77fc86890a1 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:45.294992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 827/1651] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:45.406191 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dde144-f39c-4e4a-8d37-5416578d0aca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:45.408725 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dde144-f39c-4e4a-8d37-5416578d0aca service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:45.445293 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 825/1652] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:47.523499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010f87-795f-4a3e-afaf-f77fc86890a1 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:47.525600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1a010f87-795f-4a3e-afaf-f77fc86890a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:47.593754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a010f87-795f-4a3e-afaf-f77fc86890a1 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:47.634175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:47.635233 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:47.635722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:47.636199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] PATH_INFO: `/v3/projects/b75d70430685440e869b8b40c2b2c05d` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:47.647035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b75d70430685440e869b8b40c2b2c05d)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:47.648851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: WARNING py.warnings [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:04:47.648851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: warnings.warn(msg) Aug 01 12:04:47.648851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: Aug 01 12:04:47.659516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:47.872314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b75d70430685440e869b8b40c2b2c05d', 'request_id': 'req-bef9bc5a-7a8f-47df-bdbf-9656069aa265'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:47.874670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:47.908412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b75d70430685440e869b8b40c2b2c05d', 'request_id': 'req-bef9bc5a-7a8f-47df-bdbf-9656069aa265'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:47.924072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b75d70430685440e869b8b40c2b2c05d', 'request_id': 'req-bef9bc5a-7a8f-47df-bdbf-9656069aa265'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:47.951186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] The token cache is being invalidate because project b75d70430685440e869b8b40c2b2c05d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:04:47.951738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:47.953132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-bef9bc5a-7a8f-47df-bdbf-9656069aa265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:47.956331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 828/1653] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:04:47 2022] DELETE /identity/v3/projects/b75d70430685440e869b8b40c2b2c05d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:47.960141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dde144-f39c-4e4a-8d37-5416578d0aca service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:47.979427 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dde144-f39c-4e4a-8d37-5416578d0aca service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:48.063157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:48.136185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:48.137154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:48.137875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:48.138067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] PATH_INFO: `/v3/users/381fc3b652044bf5b8943ffcf4a00849` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:48.141573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=381fc3b652044bf5b8943ffcf4a00849)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:48.143123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: WARNING py.warnings [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 01 12:04:48.143123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: warnings.warn(msg) Aug 01 12:04:48.143123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: Aug 01 12:04:48.145270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:48.239612 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d73168-c1b9-4fc1-bc06-3c8f3c1a8560 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:48.240094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d73168-c1b9-4fc1-bc06-3c8f3c1a8560 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:48.240550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d73168-c1b9-4fc1-bc06-3c8f3c1a8560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:48.309260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381fc3b652044bf5b8943ffcf4a00849'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:48.313610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381fc3b652044bf5b8943ffcf4a00849'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:48.326267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-75d73168-c1b9-4fc1-bc06-3c8f3c1a8560 None None] MFA Rules not processed for user `801ae1e600cb4aa1b84ee9cbc7cf0efb`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:48.329446 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381fc3b652044bf5b8943ffcf4a00849'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:48.329886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-75d73168-c1b9-4fc1-bc06-3c8f3c1a8560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:48.369532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381fc3b652044bf5b8943ffcf4a00849'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:48.404711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381fc3b652044bf5b8943ffcf4a00849'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:48.437208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 826/1654] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:04:47 2022] DELETE /identity/v3/users/381fc3b652044bf5b8943ffcf4a00849 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:48.509203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 829/1655] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:04:50.609829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:50.612598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:50.892176 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b2bac-36a0-4038-835a-6aec6bea0c93 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:50.903629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:50.905511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:50.905799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:50.907695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] PATH_INFO: `/v3/projects/6bffc7ee232a4a0babd0e9b6fd649624` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:50.909658 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bffc7ee232a4a0babd0e9b6fd649624)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:50.912062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:50.962246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0818c4-d268-4bcc-b5f1-61b9e41dc294 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:50.962847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0818c4-d268-4bcc-b5f1-61b9e41dc294 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:50.963247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0818c4-d268-4bcc-b5f1-61b9e41dc294 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:50.966654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 830/1656] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:04:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:04:51.014688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bffc7ee232a4a0babd0e9b6fd649624', 'request_id': 'req-d89ee3b0-0540-45c0-9d28-61c1211df088'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:51.015940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:51.025860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a1fd7-ab4c-48d8-841e-1f5c7c441fc5 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.026157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a1fd7-ab4c-48d8-841e-1f5c7c441fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.026827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a1fd7-ab4c-48d8-841e-1f5c7c441fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.048823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bffc7ee232a4a0babd0e9b6fd649624', 'request_id': 'req-d89ee3b0-0540-45c0-9d28-61c1211df088'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:51.070358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bffc7ee232a4a0babd0e9b6fd649624', 'request_id': 'req-d89ee3b0-0540-45c0-9d28-61c1211df088'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:51.091398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-668a1fd7-ab4c-48d8-841e-1f5c7c441fc5 None None] MFA Rules not processed for user `9db663d35f964d42938be0d1a9ed4071`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:51.094261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-668a1fd7-ab4c-48d8-841e-1f5c7c441fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:51.143997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] The token cache is being invalidate because project 6bffc7ee232a4a0babd0e9b6fd649624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:04:51.144640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:51.146137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-d89ee3b0-0540-45c0-9d28-61c1211df088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:51.152336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 827/1657] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:04:50 2022] DELETE /identity/v3/projects/6bffc7ee232a4a0babd0e9b6fd649624 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:51.154533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7901ae-4db6-4814-8634-fd632b17311c None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.155719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7901ae-4db6-4814-8634-fd632b17311c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.160018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7901ae-4db6-4814-8634-fd632b17311c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.241777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2f7901ae-4db6-4814-8634-fd632b17311c None None] MFA Rules not processed for user `443696f19aec46fc9cd59bd9d61587f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:51.242785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2f7901ae-4db6-4814-8634-fd632b17311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:51.313292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 831/1658] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:04:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:04:51.315622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66501e58-693a-40db-a169-42dd0f4b9618 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.315914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66501e58-693a-40db-a169-42dd0f4b9618 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.316197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66501e58-693a-40db-a169-42dd0f4b9618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.361389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-66501e58-693a-40db-a169-42dd0f4b9618 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:51.363420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-66501e58-693a-40db-a169-42dd0f4b9618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:51.418585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 828/1659] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:51.450271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400121d-f67b-4211-84e7-7d370c4c3d3d None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.450772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400121d-f67b-4211-84e7-7d370c4c3d3d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.451125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400121d-f67b-4211-84e7-7d370c4c3d3d None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.453056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 829/1660] 10.222.0.49 () {54 vars in 961 bytes} [Mon Aug 1 12:04:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:04:51.465437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad35d-0a24-41d5-8501-2ff688ed2864 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.467845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad35d-0a24-41d5-8501-2ff688ed2864 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.469618 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ad35d-0a24-41d5-8501-2ff688ed2864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.520195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:51.555542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 832/1661] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:51.557202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f748a-d2b7-4315-ad27-a4911654c864 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.557503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f748a-d2b7-4315-ad27-a4911654c864 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.557782 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f748a-d2b7-4315-ad27-a4911654c864 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.559737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a01ad35d-0a24-41d5-8501-2ff688ed2864 None None] MFA Rules not processed for user `9db663d35f964d42938be0d1a9ed4071`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:51.559871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 833/1662] 10.222.0.49 () {54 vars in 962 bytes} [Mon Aug 1 12:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:04:51.561523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b34408-35be-43a8-949e-c38f8c8a4bdd None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.561916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b34408-35be-43a8-949e-c38f8c8a4bdd None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.562286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b34408-35be-43a8-949e-c38f8c8a4bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.566117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a01ad35d-0a24-41d5-8501-2ff688ed2864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:51.593705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-19b34408-35be-43a8-949e-c38f8c8a4bdd None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:51.594957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-19b34408-35be-43a8-949e-c38f8c8a4bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:51.616564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 834/1663] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:51.621838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62e498-af5c-4b61-99d0-b48e9278722a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.622202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62e498-af5c-4b61-99d0-b48e9278722a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.622552 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62e498-af5c-4b61-99d0-b48e9278722a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.626712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 830/1664] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Aug 1 12:04:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:51.635895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ad35d-0a24-41d5-8501-2ff688ed2864 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:51.674518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ad35d-0a24-41d5-8501-2ff688ed2864 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:51.683372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39d555-5f53-453b-ae55-8ec9cdc54394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:51.684608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39d555-5f53-453b-ae55-8ec9cdc54394 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.684608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39d555-5f53-453b-ae55-8ec9cdc54394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.684810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c39d555-5f53-453b-ae55-8ec9cdc54394 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.685573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39d555-5f53-453b-ae55-8ec9cdc54394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:04:51.685948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39d555-5f53-453b-ae55-8ec9cdc54394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:51.689340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c39d555-5f53-453b-ae55-8ec9cdc54394 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:51.719795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-2d62e498-af5c-4b61-99d0-b48e9278722a None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:51.722864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2d62e498-af5c-4b61-99d0-b48e9278722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:51.745603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 831/1665] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:04:51.745603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39d555-5f53-453b-ae55-8ec9cdc54394 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:51.773783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c39d555-5f53-453b-ae55-8ec9cdc54394 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:51.780567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1eee29-5436-4f0f-86e6-d8aed4503bac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:51.781836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eee29-5436-4f0f-86e6-d8aed4503bac service placement] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.782623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eee29-5436-4f0f-86e6-d8aed4503bac service placement] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.784099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1eee29-5436-4f0f-86e6-d8aed4503bac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.809281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1eee29-5436-4f0f-86e6-d8aed4503bac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:51.811142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1eee29-5436-4f0f-86e6-d8aed4503bac service placement] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:51.837308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 832/1666] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:51.838335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1eee29-5436-4f0f-86e6-d8aed4503bac service placement] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:51.865545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1eee29-5436-4f0f-86e6-d8aed4503bac service placement] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:51.893331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f162a7-b6fb-4870-8fb3-e52f3a04fa52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:51.894569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f162a7-b6fb-4870-8fb3-e52f3a04fa52 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.895391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f162a7-b6fb-4870-8fb3-e52f3a04fa52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.896054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f162a7-b6fb-4870-8fb3-e52f3a04fa52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.899496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f162a7-b6fb-4870-8fb3-e52f3a04fa52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:51.906254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f162a7-b6fb-4870-8fb3-e52f3a04fa52 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:51.930370 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 835/1667] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Aug 1 12:04:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:51.939587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62e498-af5c-4b61-99d0-b48e9278722a None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:51.954323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62e498-af5c-4b61-99d0-b48e9278722a None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:51.969775 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87404e4-ff5d-4f6b-a43a-e7c1977a5715 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:51.971230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87404e4-ff5d-4f6b-a43a-e7c1977a5715 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:51.972082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87404e4-ff5d-4f6b-a43a-e7c1977a5715 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:51.973921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87404e4-ff5d-4f6b-a43a-e7c1977a5715 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:51.975199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 833/1668] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:51 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:51.981400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f162a7-b6fb-4870-8fb3-e52f3a04fa52 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:51.998413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87404e4-ff5d-4f6b-a43a-e7c1977a5715 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:52.002704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87404e4-ff5d-4f6b-a43a-e7c1977a5715 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:52.032006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 836/1669] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:04:52.085856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f162a7-b6fb-4870-8fb3-e52f3a04fa52 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:52.092611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96970e0-1cae-464b-85e6-26c773bcf971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:52.093752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96970e0-1cae-464b-85e6-26c773bcf971 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:52.094127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96970e0-1cae-464b-85e6-26c773bcf971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:52.094503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96970e0-1cae-464b-85e6-26c773bcf971 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:52.095297 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96970e0-1cae-464b-85e6-26c773bcf971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:52.097587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96970e0-1cae-464b-85e6-26c773bcf971 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:52.129653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 834/1670] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:51 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:52.133770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87404e4-ff5d-4f6b-a43a-e7c1977a5715 service nova] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:52.150101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87404e4-ff5d-4f6b-a43a-e7c1977a5715 service nova] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:52.165822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1594d-95f5-495b-bd5e-b9e4db9a9791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:52.166630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1594d-95f5-495b-bd5e-b9e4db9a9791 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:52.167001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1594d-95f5-495b-bd5e-b9e4db9a9791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:52.167711 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1594d-95f5-495b-bd5e-b9e4db9a9791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:52.168810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1594d-95f5-495b-bd5e-b9e4db9a9791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:52.173557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1594d-95f5-495b-bd5e-b9e4db9a9791 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:52.197230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 837/1671] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:52.201221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96970e0-1cae-464b-85e6-26c773bcf971 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:52.207494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:52.215346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96970e0-1cae-464b-85e6-26c773bcf971 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:52.238989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be577b5a-6752-4b62-9e4e-12bdf4a3fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:52.240543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be577b5a-6752-4b62-9e4e-12bdf4a3fc64 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:52.240543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be577b5a-6752-4b62-9e4e-12bdf4a3fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:52.241874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be577b5a-6752-4b62-9e4e-12bdf4a3fc64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:52.242853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be577b5a-6752-4b62-9e4e-12bdf4a3fc64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:52.248266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be577b5a-6752-4b62-9e4e-12bdf4a3fc64 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:52.271468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 835/1672] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:52.275459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1594d-95f5-495b-bd5e-b9e4db9a9791 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:52.288930 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1594d-95f5-495b-bd5e-b9e4db9a9791 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:52.302397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78893d3-95e5-4c7a-a34b-7b6da1522c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:52.303286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78893d3-95e5-4c7a-a34b-7b6da1522c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:52.303663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78893d3-95e5-4c7a-a34b-7b6da1522c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:52.304223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78893d3-95e5-4c7a-a34b-7b6da1522c2b admin admin] PATH_INFO: `/v3/projects/cbd0a7008ded4655b49f1794cc152423/users/cac1fa5a354640968271cd9dd452db15/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:52.346984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78893d3-95e5-4c7a-a34b-7b6da1522c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd0a7008ded4655b49f1794cc152423, user_id=cac1fa5a354640968271cd9dd452db15, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:52.350531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78893d3-95e5-4c7a-a34b-7b6da1522c2b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:52.372501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 838/1673] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:52 2022] PUT /identity/v3/projects/cbd0a7008ded4655b49f1794cc152423/users/cac1fa5a354640968271cd9dd452db15/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:04:52.383657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8c92c-c401-4770-8170-531d63a3b613 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:52.384181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8c92c-c401-4770-8170-531d63a3b613 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:52.384569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8c92c-c401-4770-8170-531d63a3b613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:52.449581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-09f8c92c-c401-4770-8170-531d63a3b613 None None] MFA Rules not processed for user `cac1fa5a354640968271cd9dd452db15`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:52.451009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-09f8c92c-c401-4770-8170-531d63a3b613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:52.617590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 836/1674] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:52.623201 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78893d3-95e5-4c7a-a34b-7b6da1522c2b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:52.718105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78893d3-95e5-4c7a-a34b-7b6da1522c2b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:52.723418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0157d7a-f014-4341-be04-2922b2400adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:52.724332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0157d7a-f014-4341-be04-2922b2400adc admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:52.724618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0157d7a-f014-4341-be04-2922b2400adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:52.724894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0157d7a-f014-4341-be04-2922b2400adc admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:52.726384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0157d7a-f014-4341-be04-2922b2400adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:52.727910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0157d7a-f014-4341-be04-2922b2400adc admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:52.786136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 839/1675] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:52 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:52.790025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8c92c-c401-4770-8170-531d63a3b613 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:52.831941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8c92c-c401-4770-8170-531d63a3b613 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:52.964788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ed2b9-b8d0-4687-ac24-f4ed3c3e2608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:52.965939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ed2b9-b8d0-4687-ac24-f4ed3c3e2608 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:52.966384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ed2b9-b8d0-4687-ac24-f4ed3c3e2608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:52.966947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ed2b9-b8d0-4687-ac24-f4ed3c3e2608 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:52.968229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ed2b9-b8d0-4687-ac24-f4ed3c3e2608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:52.971610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ed2b9-b8d0-4687-ac24-f4ed3c3e2608 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:53.033378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 837/1676] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:52 2022] POST /identity/v3/users => generated 291 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:04:53.039693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0157d7a-f014-4341-be04-2922b2400adc admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:53.052130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0157d7a-f014-4341-be04-2922b2400adc admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:53.074356 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4331ff8-db81-4cee-b603-10d3ce2f3fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:53.074978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4331ff8-db81-4cee-b603-10d3ce2f3fec admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:53.076730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4331ff8-db81-4cee-b603-10d3ce2f3fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:53.076730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4331ff8-db81-4cee-b603-10d3ce2f3fec admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:53.077046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4331ff8-db81-4cee-b603-10d3ce2f3fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:53.080543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4331ff8-db81-4cee-b603-10d3ce2f3fec admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:53.088592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:53.112634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 840/1677] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:53.117720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ed2b9-b8d0-4687-ac24-f4ed3c3e2608 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:53.145802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ed2b9-b8d0-4687-ac24-f4ed3c3e2608 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:53.187749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a45ae9-8f71-425c-a659-9562675ea09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:53.189872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a45ae9-8f71-425c-a659-9562675ea09d admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:53.194137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a45ae9-8f71-425c-a659-9562675ea09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:53.194555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a45ae9-8f71-425c-a659-9562675ea09d admin admin] PATH_INFO: `/v3/projects/05404d89cc6747b88fff1c751477c76f/users/6bce2fdabfa54a8d9fa0389774bbf1a4/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:53.232232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a45ae9-8f71-425c-a659-9562675ea09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05404d89cc6747b88fff1c751477c76f, user_id=6bce2fdabfa54a8d9fa0389774bbf1a4, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:53.235600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a45ae9-8f71-425c-a659-9562675ea09d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:53.261568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 838/1678] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:53 2022] PUT /identity/v3/projects/05404d89cc6747b88fff1c751477c76f/users/6bce2fdabfa54a8d9fa0389774bbf1a4/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:53.265822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4331ff8-db81-4cee-b603-10d3ce2f3fec admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:53.279139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4331ff8-db81-4cee-b603-10d3ce2f3fec admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:53.293914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a45ae9-8f71-425c-a659-9562675ea09d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:53.301793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-79a45ae9-8f71-425c-a659-9562675ea09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:53.426504 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850904d8-0617-49f2-91d3-4e32240313aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:53.427256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850904d8-0617-49f2-91d3-4e32240313aa admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:53.427540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850904d8-0617-49f2-91d3-4e32240313aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:53.427817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850904d8-0617-49f2-91d3-4e32240313aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:53.428600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850904d8-0617-49f2-91d3-4e32240313aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:53.430317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850904d8-0617-49f2-91d3-4e32240313aa admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:53.457376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 841/1679] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:04:53.458799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:53.462831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850904d8-0617-49f2-91d3-4e32240313aa admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:53.487337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850904d8-0617-49f2-91d3-4e32240313aa admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:53.493994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a45ae9-8f71-425c-a659-9562675ea09d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:53.500750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b9273-73bb-4015-a1dc-74c3032b3c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:53.501481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b9273-73bb-4015-a1dc-74c3032b3c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:53.501766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b9273-73bb-4015-a1dc-74c3032b3c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:53.502047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b9273-73bb-4015-a1dc-74c3032b3c2a admin admin] PATH_INFO: `/v3/projects/05404d89cc6747b88fff1c751477c76f/users/6bce2fdabfa54a8d9fa0389774bbf1a4/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:53.504159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b9273-73bb-4015-a1dc-74c3032b3c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05404d89cc6747b88fff1c751477c76f, user_id=6bce2fdabfa54a8d9fa0389774bbf1a4, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:53.506998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b9273-73bb-4015-a1dc-74c3032b3c2a admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:53.519935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:53.520773 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:53.521037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:53.521312 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] PATH_INFO: `/v3/users/01390dc3eacb4c08bde3e97540d0d1c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:53.524597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01390dc3eacb4c08bde3e97540d0d1c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:53.532494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:53.555929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 842/1680] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:04:53 2022] PUT /identity/v3/projects/05404d89cc6747b88fff1c751477c76f/users/6bce2fdabfa54a8d9fa0389774bbf1a4/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:53.566744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac52d3b-362d-446e-87fb-2915b75afa7d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:53.567064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac52d3b-362d-446e-87fb-2915b75afa7d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:53.571671 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac52d3b-362d-446e-87fb-2915b75afa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:53.683402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-fac52d3b-362d-446e-87fb-2915b75afa7d None None] MFA Rules not processed for user `6bce2fdabfa54a8d9fa0389774bbf1a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:53.699119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01390dc3eacb4c08bde3e97540d0d1c3'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:53.699990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01390dc3eacb4c08bde3e97540d0d1c3'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:53.706334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fac52d3b-362d-446e-87fb-2915b75afa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:53.715539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01390dc3eacb4c08bde3e97540d0d1c3'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:53.745844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01390dc3eacb4c08bde3e97540d0d1c3'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:53.765106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01390dc3eacb4c08bde3e97540d0d1c3'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:53.787135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 839/1681] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:04:53 2022] DELETE /identity/v3/users/01390dc3eacb4c08bde3e97540d0d1c3 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:04:53.949094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 843/1682] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:53 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:53.958062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:53.959810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:54.116094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf93a3-886a-4fd6-9dc1-135192258579 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:54.127625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3b9e0-a392-4735-b741-91bdadf200c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:54.129182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3b9e0-a392-4735-b741-91bdadf200c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:54.129481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3b9e0-a392-4735-b741-91bdadf200c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:54.129753 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3b9e0-a392-4735-b741-91bdadf200c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:54.275554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3b9e0-a392-4735-b741-91bdadf200c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:54.278556 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3b9e0-a392-4735-b741-91bdadf200c8 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:54.313025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 840/1683] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:53 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:54.875210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac52d3b-362d-446e-87fb-2915b75afa7d None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:54.877398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fac52d3b-362d-446e-87fb-2915b75afa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:55.075858 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac52d3b-362d-446e-87fb-2915b75afa7d None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:55.089087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:55.200826 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:55.205521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:55.206009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:55.206297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] PATH_INFO: `/v3/projects/279d5a3294934195bb335826a36aaf62` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:55.210784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=279d5a3294934195bb335826a36aaf62)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:55.219073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:55.381784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '279d5a3294934195bb335826a36aaf62', 'request_id': 'req-e44372c5-7786-4a21-9853-fd00e1b31f57'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:55.401115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '279d5a3294934195bb335826a36aaf62', 'request_id': 'req-e44372c5-7786-4a21-9853-fd00e1b31f57'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:55.426072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '279d5a3294934195bb335826a36aaf62', 'request_id': 'req-e44372c5-7786-4a21-9853-fd00e1b31f57'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:55.467180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] The token cache is being invalidate because project 279d5a3294934195bb335826a36aaf62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:04:55.467507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:55.468669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:55.474898 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 844/1684] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:04:54 2022] DELETE /identity/v3/projects/279d5a3294934195bb335826a36aaf62 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:55.479468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3b9e0-a392-4735-b741-91bdadf200c8 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:55.501354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3b9e0-a392-4735-b741-91bdadf200c8 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:55.550962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:55.708398 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:55.711398 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:55.711695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:55.711963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] PATH_INFO: `/v3/users/801ae1e600cb4aa1b84ee9cbc7cf0efb` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:55.716115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=801ae1e600cb4aa1b84ee9cbc7cf0efb)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:55.718197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:55.906816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801ae1e600cb4aa1b84ee9cbc7cf0efb'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:55.909238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801ae1e600cb4aa1b84ee9cbc7cf0efb'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:55.950004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801ae1e600cb4aa1b84ee9cbc7cf0efb'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:56.014756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801ae1e600cb4aa1b84ee9cbc7cf0efb'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:56.043045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ecceebea-40e2-4a04-8c1b-5c4861ffa3a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801ae1e600cb4aa1b84ee9cbc7cf0efb'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:56.054442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 841/1685] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:04:55 2022] DELETE /identity/v3/users/801ae1e600cb4aa1b84ee9cbc7cf0efb => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:57.372003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:57.374818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:57.402071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44372c5-7786-4a21-9853-fd00e1b31f57 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:57.547939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:57.548875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:57.549358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:57.549943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] PATH_INFO: `/v3/projects/738dc937c3df435b8a23a53bf4dfea77` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:57.551656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738dc937c3df435b8a23a53bf4dfea77)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:57.554547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:57.632442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc20fe-24a9-46bf-861e-9ff845aaec09 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:57.633288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc20fe-24a9-46bf-861e-9ff845aaec09 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:57.633622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc20fe-24a9-46bf-861e-9ff845aaec09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:57.665240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:57.671496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738dc937c3df435b8a23a53bf4dfea77', 'request_id': 'req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:57.691555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738dc937c3df435b8a23a53bf4dfea77', 'request_id': 'req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:57.709997 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738dc937c3df435b8a23a53bf4dfea77', 'request_id': 'req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:57.711426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4afc20fe-24a9-46bf-861e-9ff845aaec09 None None] MFA Rules not processed for user `9bc6d25504f1428a84ee2cd93657b9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:57.717579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4afc20fe-24a9-46bf-861e-9ff845aaec09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:57.739323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] The token cache is being invalidate because project 738dc937c3df435b8a23a53bf4dfea77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:04:57.739641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:57.742488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eee98eb5-0ef6-4a6e-8b2a-30d2a217c338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:57.750198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 845/1686] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:04:57 2022] DELETE /identity/v3/projects/738dc937c3df435b8a23a53bf4dfea77 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:57.755003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63047b-2603-4663-baf1-3d23fd38328a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:57.755407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63047b-2603-4663-baf1-3d23fd38328a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:57.755689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63047b-2603-4663-baf1-3d23fd38328a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:57.830690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-5e63047b-2603-4663-baf1-3d23fd38328a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:57.832099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5e63047b-2603-4663-baf1-3d23fd38328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:57.959024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 842/1687] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:57.959947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc20fe-24a9-46bf-861e-9ff845aaec09 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:57.972504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4afc20fe-24a9-46bf-861e-9ff845aaec09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:58.047094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 846/1688] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:58.055303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d8cda-8038-49f2-88e9-05fb343a0bdb None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:58.057281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d8cda-8038-49f2-88e9-05fb343a0bdb None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:58.057950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d8cda-8038-49f2-88e9-05fb343a0bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:58.087612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:58.130261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-753d8cda-8038-49f2-88e9-05fb343a0bdb None None] MFA Rules not processed for user `9bc6d25504f1428a84ee2cd93657b9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:58.136512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-753d8cda-8038-49f2-88e9-05fb343a0bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:58.170600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 847/1689] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:58.172698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8659d-d998-4d07-ace1-912530471446 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:58.173210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8659d-d998-4d07-ace1-912530471446 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:58.173674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8659d-d998-4d07-ace1-912530471446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:58.241238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-6cd8659d-d998-4d07-ace1-912530471446 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:58.243313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6cd8659d-d998-4d07-ace1-912530471446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:58.273739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 848/1690] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:58.275255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcb0e8-cec7-4702-bd7b-1835f288750e None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:58.275584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcb0e8-cec7-4702-bd7b-1835f288750e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:58.275892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcb0e8-cec7-4702-bd7b-1835f288750e None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:58.278583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 849/1691] 10.222.0.49 () {54 vars in 962 bytes} [Mon Aug 1 12:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:04:58.279418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcb0e8-cec7-4702-bd7b-1835f288750e None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:58.299348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcb0e8-cec7-4702-bd7b-1835f288750e None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:58.307478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc20fe-24a9-46bf-861e-9ff845aaec09 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:58.312582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7de63-af00-44ee-b3c0-4665c1a08d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:58.313314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7de63-af00-44ee-b3c0-4665c1a08d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:58.313596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7de63-af00-44ee-b3c0-4665c1a08d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:58.313870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7de63-af00-44ee-b3c0-4665c1a08d06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:58.317427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7de63-af00-44ee-b3c0-4665c1a08d06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:04:58.317897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7de63-af00-44ee-b3c0-4665c1a08d06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:58.322593 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7de63-af00-44ee-b3c0-4665c1a08d06 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:58.342725 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:58.343430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:58.343716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:58.343996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] PATH_INFO: `/v3/users/cac1fa5a354640968271cd9dd452db15` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:58.352320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac1fa5a354640968271cd9dd452db15)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:58.359948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:58.378241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 850/1692] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:04:58.379934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487c2aa-42b7-4eac-9021-6c46f36b98f9 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:58.380228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487c2aa-42b7-4eac-9021-6c46f36b98f9 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:58.380533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487c2aa-42b7-4eac-9021-6c46f36b98f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:58.452421 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d487c2aa-42b7-4eac-9021-6c46f36b98f9 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:58.462142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d487c2aa-42b7-4eac-9021-6c46f36b98f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:58.540735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac1fa5a354640968271cd9dd452db15'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:58.541747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac1fa5a354640968271cd9dd452db15'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:58.565187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac1fa5a354640968271cd9dd452db15'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:58.597962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac1fa5a354640968271cd9dd452db15'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:58.634540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac1fa5a354640968271cd9dd452db15'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:58.647446 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 843/1693] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:04:57 2022] DELETE /identity/v3/users/cac1fa5a354640968271cd9dd452db15 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:58.651540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:58.655156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:58.676270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 851/1694] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Aug 1 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:58.677374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487c2aa-42b7-4eac-9021-6c46f36b98f9 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:58.680224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d487c2aa-42b7-4eac-9021-6c46f36b98f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:58.710974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487c2aa-42b7-4eac-9021-6c46f36b98f9 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:58.784055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:58.869041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd76b-42c5-47f6-b956-c4d6b4ac0936 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:58.873622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:58.875152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:58.875446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:58.875890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] PATH_INFO: `/v3/users/e613776b2bf247c482774a55f97a3872` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:58.886057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e613776b2bf247c482774a55f97a3872)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:58.886467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9cad9-195a-4158-84a2-dd658f4da50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:58.887132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9cad9-195a-4158-84a2-dd658f4da50f admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:58.887415 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9cad9-195a-4158-84a2-dd658f4da50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:58.887700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9cad9-195a-4158-84a2-dd658f4da50f admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:58.888065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:58.888590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9cad9-195a-4158-84a2-dd658f4da50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:58.890561 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9cad9-195a-4158-84a2-dd658f4da50f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:58.982684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 844/1695] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:04:58 2022] POST /identity/v3/projects => generated 357 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:58.984560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13346c16-461e-4994-9233-d4f623bb0585 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:58.984874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13346c16-461e-4994-9233-d4f623bb0585 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:58.985148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13346c16-461e-4994-9233-d4f623bb0585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:58.998561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e613776b2bf247c482774a55f97a3872'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:59.019481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e613776b2bf247c482774a55f97a3872'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:59.031375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e613776b2bf247c482774a55f97a3872'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:59.032586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e613776b2bf247c482774a55f97a3872'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:59.033947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e613776b2bf247c482774a55f97a3872'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:04:59.042021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-13346c16-461e-4994-9233-d4f623bb0585 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:59.043928 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-13346c16-461e-4994-9233-d4f623bb0585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:59.069005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 852/1696] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:04:58 2022] DELETE /identity/v3/users/e613776b2bf247c482774a55f97a3872 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:04:59.070256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:59.074342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:59.108041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:59.256762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 845/1697] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:59.256762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13346c16-461e-4994-9233-d4f623bb0585 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:59.259681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-13346c16-461e-4994-9233-d4f623bb0585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:59.291197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13346c16-461e-4994-9233-d4f623bb0585 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:59.332727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335771c-dcf5-41cf-9a43-df43980c1805 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:59.360761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96105bf9-e5fb-4e4e-8b6e-70c2c1d06937 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:59.361625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96105bf9-e5fb-4e4e-8b6e-70c2c1d06937 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:59.361916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96105bf9-e5fb-4e4e-8b6e-70c2c1d06937 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:59.364144 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96105bf9-e5fb-4e4e-8b6e-70c2c1d06937 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:59.366328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-96105bf9-e5fb-4e4e-8b6e-70c2c1d06937 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:59.455710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dbb179-6bb4-4d1d-886b-429225a60efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:59.457414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb179-6bb4-4d1d-886b-429225a60efd admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:59.460058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb179-6bb4-4d1d-886b-429225a60efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:59.460058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dbb179-6bb4-4d1d-886b-429225a60efd admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:59.460534 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dbb179-6bb4-4d1d-886b-429225a60efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:59.462562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dbb179-6bb4-4d1d-886b-429225a60efd admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:59.506574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 846/1698] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:04:59 2022] POST /identity/v3/users => generated 279 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:04:59.509004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16002-3d18-4565-b868-be3c8d64af67 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:59.509314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16002-3d18-4565-b868-be3c8d64af67 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:59.510222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16002-3d18-4565-b868-be3c8d64af67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:59.539000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a0b16002-3d18-4565-b868-be3c8d64af67 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:59.542149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a0b16002-3d18-4565-b868-be3c8d64af67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:59.557201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 847/1699] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:59.560360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b16002-3d18-4565-b868-be3c8d64af67 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:59.564688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:04:59.566898 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a0b16002-3d18-4565-b868-be3c8d64af67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:59.571866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96105bf9-e5fb-4e4e-8b6e-70c2c1d06937 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:59.579734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96105bf9-e5fb-4e4e-8b6e-70c2c1d06937 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:59.647147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 853/1700] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:04:59.653476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb614d4-d215-4064-903e-dd7ef350479b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:59.667277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb614d4-d215-4064-903e-dd7ef350479b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:59.669189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb614d4-d215-4064-903e-dd7ef350479b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:59.768040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3eb614d4-d215-4064-903e-dd7ef350479b None None] MFA Rules not processed for user `443696f19aec46fc9cd59bd9d61587f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:04:59.772815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3eb614d4-d215-4064-903e-dd7ef350479b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:04:59.940261 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b16002-3d18-4565-b868-be3c8d64af67 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:04:59.947654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aab6cd-0f16-4bf6-9dae-3dcc84b0e0ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:04:59.948464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aab6cd-0f16-4bf6-9dae-3dcc84b0e0ed service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:04:59.948812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aab6cd-0f16-4bf6-9dae-3dcc84b0e0ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:04:59.949151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aab6cd-0f16-4bf6-9dae-3dcc84b0e0ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:04:59.960472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 854/1701] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:04:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:04:59.960472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb614d4-d215-4064-903e-dd7ef350479b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:04:59.973295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aab6cd-0f16-4bf6-9dae-3dcc84b0e0ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:04:59.975348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aab6cd-0f16-4bf6-9dae-3dcc84b0e0ed service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:04:59.993906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb614d4-d215-4064-903e-dd7ef350479b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:00.027868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 848/1702] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:04:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:00.030490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aab6cd-0f16-4bf6-9dae-3dcc84b0e0ed service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:00.031778 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec3f4-b73b-4b23-8563-c0b0249c2a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:00.032485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec3f4-b73b-4b23-8563-c0b0249c2a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.032825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec3f4-b73b-4b23-8563-c0b0249c2a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.033104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec3f4-b73b-4b23-8563-c0b0249c2a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:00.033835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ec3f4-b73b-4b23-8563-c0b0249c2a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:00.045591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ec3f4-b73b-4b23-8563-c0b0249c2a28 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:00.085502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 855/1703] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:00.086488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec3f4-b73b-4b23-8563-c0b0249c2a28 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:00.092064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aab6cd-0f16-4bf6-9dae-3dcc84b0e0ed service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:00.134501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c6767-d999-421c-be32-8c52691877cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:00.134747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec3f4-b73b-4b23-8563-c0b0249c2a28 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:00.135154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c6767-d999-421c-be32-8c52691877cc service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.135433 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c6767-d999-421c-be32-8c52691877cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.135705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c6767-d999-421c-be32-8c52691877cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:00.148196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6414cd-3f1c-4be7-b0d6-f5aad8d96845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:00.150902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6414cd-3f1c-4be7-b0d6-f5aad8d96845 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.151203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6414cd-3f1c-4be7-b0d6-f5aad8d96845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.151491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6414cd-3f1c-4be7-b0d6-f5aad8d96845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:00.152243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6414cd-3f1c-4be7-b0d6-f5aad8d96845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:00.158579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6414cd-3f1c-4be7-b0d6-f5aad8d96845 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:00.190078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 856/1704] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:00.190942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3312b-3c7e-4ac6-8789-45a87af0e106 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.192816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3312b-3c7e-4ac6-8789-45a87af0e106 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.194071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3312b-3c7e-4ac6-8789-45a87af0e106 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:00.201924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c6767-d999-421c-be32-8c52691877cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:00.203542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 857/1705] 10.222.0.49 () {54 vars in 912 bytes} [Mon Aug 1 12:05:00 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:05:00.203839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c6767-d999-421c-be32-8c52691877cc service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:00.205567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3312b-3c7e-4ac6-8789-45a87af0e106 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:00.227174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 849/1706] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:00.229033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301d670-ee26-4fd7-a848-84338781dc08 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.229328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301d670-ee26-4fd7-a848-84338781dc08 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.229612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301d670-ee26-4fd7-a848-84338781dc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:00.242697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3312b-3c7e-4ac6-8789-45a87af0e106 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:00.255362 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dea981-4d39-4e58-8c59-b0ae84bf5d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:00.256500 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dea981-4d39-4e58-8c59-b0ae84bf5d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.257210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dea981-4d39-4e58-8c59-b0ae84bf5d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.257210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dea981-4d39-4e58-8c59-b0ae84bf5d83 admin admin] PATH_INFO: `/v3/projects/9696f4282dad423cb8e0bb53ac931bb9/users/49130731dcdb45bdac56b801318c6ea9/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:00.287287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d301d670-ee26-4fd7-a848-84338781dc08 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:00.289224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d301d670-ee26-4fd7-a848-84338781dc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:00.289732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dea981-4d39-4e58-8c59-b0ae84bf5d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9696f4282dad423cb8e0bb53ac931bb9, user_id=49130731dcdb45bdac56b801318c6ea9, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:00.292949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dea981-4d39-4e58-8c59-b0ae84bf5d83 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:00.327300 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 858/1707] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:00 2022] PUT /identity/v3/projects/9696f4282dad423cb8e0bb53ac931bb9/users/49130731dcdb45bdac56b801318c6ea9/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:00.329239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b9dea-3e7b-4535-939a-c7b33d8c87c0 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.329533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b9dea-3e7b-4535-939a-c7b33d8c87c0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.329786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 850/1708] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:05:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:00.329884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b9dea-3e7b-4535-939a-c7b33d8c87c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:00.356440 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301d670-ee26-4fd7-a848-84338781dc08 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:00.362989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d301d670-ee26-4fd7-a848-84338781dc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:00.406098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-2d7b9dea-3e7b-4535-939a-c7b33d8c87c0 None None] MFA Rules not processed for user `49130731dcdb45bdac56b801318c6ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:00.410180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2d7b9dea-3e7b-4535-939a-c7b33d8c87c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:00.489830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:00.618002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301d670-ee26-4fd7-a848-84338781dc08 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:00.633962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 859/1709] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:00 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:00.636077 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd8575-a26c-4037-bd2a-b6baa2766adb None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.636395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd8575-a26c-4037-bd2a-b6baa2766adb None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.636674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd8575-a26c-4037-bd2a-b6baa2766adb None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:00.638832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 860/1710] 10.222.0.49 () {54 vars in 912 bytes} [Mon Aug 1 12:05:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:05:00.639842 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd8575-a26c-4037-bd2a-b6baa2766adb None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:00.776952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:00.803271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd8575-a26c-4037-bd2a-b6baa2766adb None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:00.813038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4b2beb-358f-48a2-8e35-25ec7b861bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:00.814326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b2beb-358f-48a2-8e35-25ec7b861bca admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.815643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b2beb-358f-48a2-8e35-25ec7b861bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.815643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4b2beb-358f-48a2-8e35-25ec7b861bca admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:00.817112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b2beb-358f-48a2-8e35-25ec7b861bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:00.823196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4b2beb-358f-48a2-8e35-25ec7b861bca admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:00.882151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642c4e3-77c6-424f-af47-aa43ff5113a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:00.883175 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642c4e3-77c6-424f-af47-aa43ff5113a4 service glance] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.883659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642c4e3-77c6-424f-af47-aa43ff5113a4 service glance] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.884094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642c4e3-77c6-424f-af47-aa43ff5113a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:00.935427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 861/1711] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:00 2022] POST /identity/v3/projects => generated 357 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:00.941074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76fd63-5801-4fb0-a623-806d548f5340 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:00.941502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76fd63-5801-4fb0-a623-806d548f5340 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:00.947420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76fd63-5801-4fb0-a623-806d548f5340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:01.021607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-9e76fd63-5801-4fb0-a623-806d548f5340 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:01.025291 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9e76fd63-5801-4fb0-a623-806d548f5340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:01.197268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642c4e3-77c6-424f-af47-aa43ff5113a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:01.199725 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c642c4e3-77c6-424f-af47-aa43ff5113a4 service glance] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:01.256014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 851/1712] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 887 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:05:01.256014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642c4e3-77c6-424f-af47-aa43ff5113a4 service glance] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:01.275036 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642c4e3-77c6-424f-af47-aa43ff5113a4 service glance] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:01.340410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 862/1713] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:05:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:01.362635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32655c2b-cbc9-4938-a173-9d1937a46304 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:01.362939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32655c2b-cbc9-4938-a173-9d1937a46304 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:01.363305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32655c2b-cbc9-4938-a173-9d1937a46304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:01.475416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-32655c2b-cbc9-4938-a173-9d1937a46304 None None] MFA Rules not processed for user `e98a16b117294e10b9d0c8c721d8e114`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:01.479481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-32655c2b-cbc9-4938-a173-9d1937a46304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:01.545448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ed3e3-04ec-4117-ba1d-0e0f932b0276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:01.546504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed3e3-04ec-4117-ba1d-0e0f932b0276 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:01.546504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed3e3-04ec-4117-ba1d-0e0f932b0276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:01.546720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ed3e3-04ec-4117-ba1d-0e0f932b0276 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:01.547687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ed3e3-04ec-4117-ba1d-0e0f932b0276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:01.549602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ed3e3-04ec-4117-ba1d-0e0f932b0276 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:01.586886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 852/1714] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:01 2022] POST /identity/v3/users => generated 285 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:01.586886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ed3e3-04ec-4117-ba1d-0e0f932b0276 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:01.593477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-230ed3e3-04ec-4117-ba1d-0e0f932b0276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:01.622213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:01.634471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ed3e3-04ec-4117-ba1d-0e0f932b0276 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:01.644564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:01.645317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:01.645608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:01.645888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] PATH_INFO: `/v3/projects/cbd0a7008ded4655b49f1794cc152423` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:01.647122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd0a7008ded4655b49f1794cc152423)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:01.648469 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 863/1715] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:01.650250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32655c2b-cbc9-4938-a173-9d1937a46304 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:01.652428 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-32655c2b-cbc9-4938-a173-9d1937a46304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:01.663679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:01.721341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32655c2b-cbc9-4938-a173-9d1937a46304 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:01.744058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:01.746378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:01.749879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:01.751384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] PATH_INFO: `/v3/projects/f10697a6130a4f2b9c0d4e5db1897d8a` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:01.756491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10697a6130a4f2b9c0d4e5db1897d8a)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:01.762373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:01.769199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd0a7008ded4655b49f1794cc152423', 'request_id': 'req-cd2311bb-beac-4636-9def-6d585d6481d6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:01.783199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd0a7008ded4655b49f1794cc152423', 'request_id': 'req-cd2311bb-beac-4636-9def-6d585d6481d6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:01.795752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd0a7008ded4655b49f1794cc152423', 'request_id': 'req-cd2311bb-beac-4636-9def-6d585d6481d6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:01.841280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] The token cache is being invalidate because project cbd0a7008ded4655b49f1794cc152423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:01.843604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:01.845453 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:01.851233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 853/1716] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:01 2022] DELETE /identity/v3/projects/cbd0a7008ded4655b49f1794cc152423 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:01.852252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:01.856011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:01.903690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:01.914688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10697a6130a4f2b9c0d4e5db1897d8a', 'request_id': 'req-393d0de3-5565-4dcc-9a86-af31f0033ad3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:01.932586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10697a6130a4f2b9c0d4e5db1897d8a', 'request_id': 'req-393d0de3-5565-4dcc-9a86-af31f0033ad3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:01.955168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10697a6130a4f2b9c0d4e5db1897d8a', 'request_id': 'req-393d0de3-5565-4dcc-9a86-af31f0033ad3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.004753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] The token cache is being invalidate because project f10697a6130a4f2b9c0d4e5db1897d8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:02.006384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.009229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.019525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 864/1717] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:01 2022] DELETE /identity/v3/projects/f10697a6130a4f2b9c0d4e5db1897d8a => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:02.020661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:02.030742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:02.052358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2311bb-beac-4636-9def-6d585d6481d6 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:02.194465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0522d4-9a33-4bc6-8938-102a23ef9ac4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:02.195639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0522d4-9a33-4bc6-8938-102a23ef9ac4 service glance] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:02.196088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0522d4-9a33-4bc6-8938-102a23ef9ac4 service glance] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:02.196555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0522d4-9a33-4bc6-8938-102a23ef9ac4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:02.201904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0c0522d4-9a33-4bc6-8938-102a23ef9ac4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:02.241862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d0de3-5565-4dcc-9a86-af31f0033ad3 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:02.254288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d77168-b308-48c4-ad89-0d028d57bcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:02.255198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d77168-b308-48c4-ad89-0d028d57bcc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:02.256870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d77168-b308-48c4-ad89-0d028d57bcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:02.257230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d77168-b308-48c4-ad89-0d028d57bcc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:02.259501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d77168-b308-48c4-ad89-0d028d57bcc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:02.264777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d77168-b308-48c4-ad89-0d028d57bcc6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:02.289736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 865/1718] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:02 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:02.292221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d77168-b308-48c4-ad89-0d028d57bcc6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:02.304221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-38d77168-b308-48c4-ad89-0d028d57bcc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:02.338677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d77168-b308-48c4-ad89-0d028d57bcc6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:02.344324 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:02.346381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:02.348182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:02.348182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:02.348182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] PATH_INFO: `/v3/users/6bce2fdabfa54a8d9fa0389774bbf1a4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:02.351434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bce2fdabfa54a8d9fa0389774bbf1a4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:02.356221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:02.393844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0522d4-9a33-4bc6-8938-102a23ef9ac4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:02.395881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0522d4-9a33-4bc6-8938-102a23ef9ac4 service glance] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:02.443193 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 854/1719] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:02.447883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0522d4-9a33-4bc6-8938-102a23ef9ac4 service glance] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:02.452448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0c0522d4-9a33-4bc6-8938-102a23ef9ac4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:02.463467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bce2fdabfa54a8d9fa0389774bbf1a4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.480341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bce2fdabfa54a8d9fa0389774bbf1a4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.504893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0522d4-9a33-4bc6-8938-102a23ef9ac4 service glance] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:02.512444 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:02.513587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:02.513976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:02.514399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] PATH_INFO: `/v3/users/5e39fd4906a645c490f320ea3c09664f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:02.516008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e39fd4906a645c490f320ea3c09664f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:02.518615 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:02.521659 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bce2fdabfa54a8d9fa0389774bbf1a4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.523885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bce2fdabfa54a8d9fa0389774bbf1a4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.526005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bce2fdabfa54a8d9fa0389774bbf1a4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.560678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 866/1720] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:02 2022] DELETE /identity/v3/users/6bce2fdabfa54a8d9fa0389774bbf1a4 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:02.561791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:02.568894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:02.620334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e39fd4906a645c490f320ea3c09664f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.621125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e39fd4906a645c490f320ea3c09664f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.637355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e39fd4906a645c490f320ea3c09664f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.673156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:02.679533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e39fd4906a645c490f320ea3c09664f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.706308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-51833b68-2f62-4ab9-ba3c-59c892bf46f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e39fd4906a645c490f320ea3c09664f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:02.717156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62faf257-552d-47a6-adcf-8af11711762b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:02.726294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 855/1721] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:02 2022] DELETE /identity/v3/users/5e39fd4906a645c490f320ea3c09664f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:02.733047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b3a3e-db87-4a3e-8f35-18099d80102b None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:02.733490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b3a3e-db87-4a3e-8f35-18099d80102b None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:02.733782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b3a3e-db87-4a3e-8f35-18099d80102b None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:02.737869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 856/1722] 10.222.0.49 () {54 vars in 912 bytes} [Mon Aug 1 12:05:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:05:02.742913 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293aca15-dce2-4c9d-bcdd-639bd88d7d64 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:02.743205 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293aca15-dce2-4c9d-bcdd-639bd88d7d64 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:02.744787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293aca15-dce2-4c9d-bcdd-639bd88d7d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:02.845320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-293aca15-dce2-4c9d-bcdd-639bd88d7d64 None None] MFA Rules not processed for user `fc8a5fa04b6141329778221169afdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:02.846613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-293aca15-dce2-4c9d-bcdd-639bd88d7d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:02.893272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2d681-5649-45c0-aed8-283a211a4f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:02.893272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d681-5649-45c0-aed8-283a211a4f6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:02.893272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d681-5649-45c0-aed8-283a211a4f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:02.893272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d681-5649-45c0-aed8-283a211a4f6b admin admin] PATH_INFO: `/v3/projects/beb17525c94443b8b747fbcd7aa53f33/users/ddf66429b45a40cc82a64c35163ed854/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:02.915091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2d681-5649-45c0-aed8-283a211a4f6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb17525c94443b8b747fbcd7aa53f33, user_id=ddf66429b45a40cc82a64c35163ed854, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:02.918201 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2d681-5649-45c0-aed8-283a211a4f6b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:02.945827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 867/1723] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:02 2022] PUT /identity/v3/projects/beb17525c94443b8b747fbcd7aa53f33/users/ddf66429b45a40cc82a64c35163ed854/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:02.951847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060ffbc-11a7-445f-acbd-e60a9b4a5b80 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:02.952147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060ffbc-11a7-445f-acbd-e60a9b4a5b80 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:02.952433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0060ffbc-11a7-445f-acbd-e60a9b4a5b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:03.016945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-0060ffbc-11a7-445f-acbd-e60a9b4a5b80 None None] MFA Rules not processed for user `8dfbda2838b242558c885f913fafb5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:03.018310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0060ffbc-11a7-445f-acbd-e60a9b4a5b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:03.035792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 857/1724] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:03.046249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293aca15-dce2-4c9d-bcdd-639bd88d7d64 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:03.053781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-293aca15-dce2-4c9d-bcdd-639bd88d7d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:03.082801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293aca15-dce2-4c9d-bcdd-639bd88d7d64 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:03.104413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 868/1725] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:05:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:03.106505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aae7c9-5da1-4c33-ae4c-1b50b6c68a18 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:03.107004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aae7c9-5da1-4c33-ae4c-1b50b6c68a18 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:03.107341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aae7c9-5da1-4c33-ae4c-1b50b6c68a18 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:03.111878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 869/1726] 10.222.0.49 () {54 vars in 912 bytes} [Mon Aug 1 12:05:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:05:03.114822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcae557-49dd-4271-ac0d-d1c580c652c6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:03.116327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcae557-49dd-4271-ac0d-d1c580c652c6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:03.124750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcae557-49dd-4271-ac0d-d1c580c652c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:03.196554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-abcae557-49dd-4271-ac0d-d1c580c652c6 None None] MFA Rules not processed for user `fc8a5fa04b6141329778221169afdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:03.201819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-abcae557-49dd-4271-ac0d-d1c580c652c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:03.319728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:03.430584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8bc59-5041-4112-86cb-75233510fef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:03.430690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8bc59-5041-4112-86cb-75233510fef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:03.431007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8bc59-5041-4112-86cb-75233510fef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:03.431229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8bc59-5041-4112-86cb-75233510fef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:03.438282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8bc59-5041-4112-86cb-75233510fef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:03.448263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8bc59-5041-4112-86cb-75233510fef5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:03.495258 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 858/1727] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:03.503591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8bc59-5041-4112-86cb-75233510fef5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:03.511468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-27f8bc59-5041-4112-86cb-75233510fef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:03.525620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 870/1728] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:03.527748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f491a6-be25-45f1-917e-2cd6483e3216 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:03.528413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f491a6-be25-45f1-917e-2cd6483e3216 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:03.528913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f491a6-be25-45f1-917e-2cd6483e3216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:03.549188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8bc59-5041-4112-86cb-75233510fef5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:03.564974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c1ddc-6ec6-4f29-bd97-6b53f00a0cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:03.565646 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c1ddc-6ec6-4f29-bd97-6b53f00a0cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:03.565925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c1ddc-6ec6-4f29-bd97-6b53f00a0cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:03.566196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c1ddc-6ec6-4f29-bd97-6b53f00a0cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:03.598471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-43f491a6-be25-45f1-917e-2cd6483e3216 None None] MFA Rules not processed for user `8dfbda2838b242558c885f913fafb5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:03.605539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-43f491a6-be25-45f1-917e-2cd6483e3216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:03.609076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c1ddc-6ec6-4f29-bd97-6b53f00a0cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:03.630795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c1ddc-6ec6-4f29-bd97-6b53f00a0cd1 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:03.631801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 871/1729] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:05:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:03.638034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f491a6-be25-45f1-917e-2cd6483e3216 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:03.647423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 859/1730] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:03.654448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c1ddc-6ec6-4f29-bd97-6b53f00a0cd1 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:03.655932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-515c1ddc-6ec6-4f29-bd97-6b53f00a0cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:03.671566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f491a6-be25-45f1-917e-2cd6483e3216 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:03.684145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d0dca-609e-4ed1-b607-aa87cdec28fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:03.684880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0dca-609e-4ed1-b607-aa87cdec28fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:03.684880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0dca-609e-4ed1-b607-aa87cdec28fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:03.685067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8d0dca-609e-4ed1-b607-aa87cdec28fa admin admin] PATH_INFO: `/v3/projects/beb17525c94443b8b747fbcd7aa53f33/users/ddf66429b45a40cc82a64c35163ed854/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:03.688596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d0dca-609e-4ed1-b607-aa87cdec28fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=beb17525c94443b8b747fbcd7aa53f33, user_id=ddf66429b45a40cc82a64c35163ed854, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:03.690880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c1ddc-6ec6-4f29-bd97-6b53f00a0cd1 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:03.696606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8d0dca-609e-4ed1-b607-aa87cdec28fa admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:03.703240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:03.703914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:03.704280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:03.704661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:03.704949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] PATH_INFO: `/v3/projects/05404d89cc6747b88fff1c751477c76f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:03.708806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05404d89cc6747b88fff1c751477c76f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:03.714831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:03.733369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 872/1731] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:03 2022] PUT /identity/v3/projects/beb17525c94443b8b747fbcd7aa53f33/users/ddf66429b45a40cc82a64c35163ed854/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:03.734587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d0dca-609e-4ed1-b607-aa87cdec28fa admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:03.929627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8d0dca-609e-4ed1-b607-aa87cdec28fa admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:03.941670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05404d89cc6747b88fff1c751477c76f', 'request_id': 'req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:03.965203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05404d89cc6747b88fff1c751477c76f', 'request_id': 'req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:03.982708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05404d89cc6747b88fff1c751477c76f', 'request_id': 'req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:04.019973 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] The token cache is being invalidate because project 05404d89cc6747b88fff1c751477c76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:04.020551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:04.021562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ad0ca4c6-1bcc-4a93-945e-7eddf5175ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:04.026799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 860/1732] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:03 2022] DELETE /identity/v3/projects/05404d89cc6747b88fff1c751477c76f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:04.028423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1ba2f-6311-4f41-b0d4-246518692b11 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:04.028641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1ba2f-6311-4f41-b0d4-246518692b11 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:04.028919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1ba2f-6311-4f41-b0d4-246518692b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:04.087243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4809c3-a922-44bd-9e09-cd274120bfa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:04.087915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4809c3-a922-44bd-9e09-cd274120bfa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:04.088195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4809c3-a922-44bd-9e09-cd274120bfa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:04.088494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4809c3-a922-44bd-9e09-cd274120bfa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:04.092483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8a4809c3-a922-44bd-9e09-cd274120bfa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:04.120011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7ba1ba2f-6311-4f41-b0d4-246518692b11 None None] MFA Rules not processed for user `ddf66429b45a40cc82a64c35163ed854`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:04.121330 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7ba1ba2f-6311-4f41-b0d4-246518692b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:04.133265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4809c3-a922-44bd-9e09-cd274120bfa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:04.137704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4809c3-a922-44bd-9e09-cd274120bfa3 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:04.164415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 873/1733] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:04.165735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3b4db-ebaa-4756-98cc-d94fbfae52bd None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:04.166014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3b4db-ebaa-4756-98cc-d94fbfae52bd None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:04.170173 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3b4db-ebaa-4756-98cc-d94fbfae52bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:04.221835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-03c3b4db-ebaa-4756-98cc-d94fbfae52bd None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:04.223954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-03c3b4db-ebaa-4756-98cc-d94fbfae52bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:04.304177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 861/1734] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:04 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:04.316540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1ba2f-6311-4f41-b0d4-246518692b11 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:04.318364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7ba1ba2f-6311-4f41-b0d4-246518692b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:04.389462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 874/1735] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:04.398358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9ea11-2636-4a86-a40e-407b25890800 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:04.398679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9ea11-2636-4a86-a40e-407b25890800 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:04.398914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9ea11-2636-4a86-a40e-407b25890800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:04.445157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:04.445991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-86d9ea11-2636-4a86-a40e-407b25890800 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:04.447379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-86d9ea11-2636-4a86-a40e-407b25890800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:04.472703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 875/1736] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:04.476867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9ea11-2636-4a86-a40e-407b25890800 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:04.482874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1ba2f-6311-4f41-b0d4-246518692b11 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:04.494406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35838c-feb4-446f-b2f7-9c4409f9e572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:04.495306 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35838c-feb4-446f-b2f7-9c4409f9e572 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:04.495859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35838c-feb4-446f-b2f7-9c4409f9e572 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:04.496413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35838c-feb4-446f-b2f7-9c4409f9e572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:04.500397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9ea11-2636-4a86-a40e-407b25890800 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:04.510246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35838c-feb4-446f-b2f7-9c4409f9e572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:04.512126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35838c-feb4-446f-b2f7-9c4409f9e572 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:04.513648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd44bfb-a0e1-46f5-8bdc-fb8f8fe504e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:04.514525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd44bfb-a0e1-46f5-8bdc-fb8f8fe504e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:04.514825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd44bfb-a0e1-46f5-8bdc-fb8f8fe504e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:04.515094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd44bfb-a0e1-46f5-8bdc-fb8f8fe504e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:04.515820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd44bfb-a0e1-46f5-8bdc-fb8f8fe504e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:04.516128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd44bfb-a0e1-46f5-8bdc-fb8f8fe504e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:04.518529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd44bfb-a0e1-46f5-8bdc-fb8f8fe504e7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:04.531678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 862/1737] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:04 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:05:04.550521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35838c-feb4-446f-b2f7-9c4409f9e572 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:04.551244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 876/1738] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:05:04.577054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35838c-feb4-446f-b2f7-9c4409f9e572 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:04.597053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b4b14-5338-400a-bbdb-628863830cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:04.597615 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b4b14-5338-400a-bbdb-628863830cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:04.597900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b4b14-5338-400a-bbdb-628863830cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:04.598187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b4b14-5338-400a-bbdb-628863830cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:04.599250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b4b14-5338-400a-bbdb-628863830cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:04.600947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b4b14-5338-400a-bbdb-628863830cd1 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:04.601481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd44bfb-a0e1-46f5-8bdc-fb8f8fe504e7 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:04.607845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5dd44bfb-a0e1-46f5-8bdc-fb8f8fe504e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:04.684328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 863/1739] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:04 2022] POST /identity/v3/projects => generated 401 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:04.688875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b4b14-5338-400a-bbdb-628863830cd1 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:04.692633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd44bfb-a0e1-46f5-8bdc-fb8f8fe504e7 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:04.805018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:04.905374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:04.905585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:04.905839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:04.906177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] PATH_INFO: `/v3/projects/eac9fa5c4c564bbfa256b9f28da9ee52` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:04.906314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b4b14-5338-400a-bbdb-628863830cd1 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:04.908062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eac9fa5c4c564bbfa256b9f28da9ee52)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:04.910262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:04.920502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d64d12-e50e-420f-bdb2-50069830bfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:04.923002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d64d12-e50e-420f-bdb2-50069830bfff admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:04.923448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d64d12-e50e-420f-bdb2-50069830bfff admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:04.923937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d64d12-e50e-420f-bdb2-50069830bfff admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:04.925744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d64d12-e50e-420f-bdb2-50069830bfff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:04.932316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d64d12-e50e-420f-bdb2-50069830bfff admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:04.999334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 864/1740] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:04 2022] POST /identity/v3/users => generated 301 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:05.000802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d64d12-e50e-420f-bdb2-50069830bfff admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:05.107111 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eac9fa5c4c564bbfa256b9f28da9ee52', 'request_id': 'req-47d4b002-9c36-4d32-8668-b610c6bdff99'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:05.142866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eac9fa5c4c564bbfa256b9f28da9ee52', 'request_id': 'req-47d4b002-9c36-4d32-8668-b610c6bdff99'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:05.176081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eac9fa5c4c564bbfa256b9f28da9ee52', 'request_id': 'req-47d4b002-9c36-4d32-8668-b610c6bdff99'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:05.229887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] The token cache is being invalidate because project eac9fa5c4c564bbfa256b9f28da9ee52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:05.233218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:05.235764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:05.245373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 877/1741] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:04 2022] DELETE /identity/v3/projects/eac9fa5c4c564bbfa256b9f28da9ee52 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:05.247154 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:05.254879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:05.303351 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d64d12-e50e-420f-bdb2-50069830bfff admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:05.394020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4b002-9c36-4d32-8668-b610c6bdff99 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:05.399600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58833663-5ce4-4331-9645-c18b841f7d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:05.400695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58833663-5ce4-4331-9645-c18b841f7d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:05.401162 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58833663-5ce4-4331-9645-c18b841f7d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:05.401534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58833663-5ce4-4331-9645-c18b841f7d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:05.402466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58833663-5ce4-4331-9645-c18b841f7d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:05.404250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58833663-5ce4-4331-9645-c18b841f7d17 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:05.420186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 878/1742] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:05.421516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58833663-5ce4-4331-9645-c18b841f7d17 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:05.423421 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-58833663-5ce4-4331-9645-c18b841f7d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:05.469648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faac6a8-e238-4cd4-8cbf-26bb2564eef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:05.473648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faac6a8-e238-4cd4-8cbf-26bb2564eef9 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:05.476630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faac6a8-e238-4cd4-8cbf-26bb2564eef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:05.476950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faac6a8-e238-4cd4-8cbf-26bb2564eef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:05.485282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2faac6a8-e238-4cd4-8cbf-26bb2564eef9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:05.526455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:05.563641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:05.617563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58833663-5ce4-4331-9645-c18b841f7d17 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:05.625784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8919e32-5ddb-4743-a200-36df5ae17899 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:05.626445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8919e32-5ddb-4743-a200-36df5ae17899 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:05.626744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8919e32-5ddb-4743-a200-36df5ae17899 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:05.627087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8919e32-5ddb-4743-a200-36df5ae17899 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:05.641991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e8919e32-5ddb-4743-a200-36df5ae17899 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:05.642720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faac6a8-e238-4cd4-8cbf-26bb2564eef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:05.645968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faac6a8-e238-4cd4-8cbf-26bb2564eef9 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:05.668488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 865/1743] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:04 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:05.671591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faac6a8-e238-4cd4-8cbf-26bb2564eef9 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:05.675446 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2faac6a8-e238-4cd4-8cbf-26bb2564eef9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:05.712253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faac6a8-e238-4cd4-8cbf-26bb2564eef9 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:05.739772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81b774-b16d-4616-9bf0-4a2259fb2606 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:05.740521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81b774-b16d-4616-9bf0-4a2259fb2606 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:05.740812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81b774-b16d-4616-9bf0-4a2259fb2606 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:05.741138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81b774-b16d-4616-9bf0-4a2259fb2606 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:05.743947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6a81b774-b16d-4616-9bf0-4a2259fb2606 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:05.782947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8919e32-5ddb-4743-a200-36df5ae17899 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:05.784822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8919e32-5ddb-4743-a200-36df5ae17899 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:05.815634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 879/1744] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:05.818704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb34ad-43f8-48c2-b39e-826c4b636525 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:05.819239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb34ad-43f8-48c2-b39e-826c4b636525 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:05.819616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb34ad-43f8-48c2-b39e-826c4b636525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:05.883637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c3eb34ad-43f8-48c2-b39e-826c4b636525 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:05.888769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c3eb34ad-43f8-48c2-b39e-826c4b636525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:05.897598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81b774-b16d-4616-9bf0-4a2259fb2606 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:05.899708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81b774-b16d-4616-9bf0-4a2259fb2606 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:05.908366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 880/1745] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:05.909926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb34ad-43f8-48c2-b39e-826c4b636525 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:05.915134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c3eb34ad-43f8-48c2-b39e-826c4b636525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:05.920197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 866/1746] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:05.921330 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81b774-b16d-4616-9bf0-4a2259fb2606 service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:05.935968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6a81b774-b16d-4616-9bf0-4a2259fb2606 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:06.142877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb34ad-43f8-48c2-b39e-826c4b636525 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:06.145462 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81b774-b16d-4616-9bf0-4a2259fb2606 service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:06.155425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26475a3b-1804-4b96-88e9-60759c8fd5cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:06.155425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26475a3b-1804-4b96-88e9-60759c8fd5cb service placement] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:06.155847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26475a3b-1804-4b96-88e9-60759c8fd5cb service placement] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:06.156239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26475a3b-1804-4b96-88e9-60759c8fd5cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:06.161933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-26475a3b-1804-4b96-88e9-60759c8fd5cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:06.163962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046717bd-ae38-4dbb-a0b3-4f40fc5c1a15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:06.164686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046717bd-ae38-4dbb-a0b3-4f40fc5c1a15 service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:06.164974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046717bd-ae38-4dbb-a0b3-4f40fc5c1a15 service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:06.165482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046717bd-ae38-4dbb-a0b3-4f40fc5c1a15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:06.190614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26475a3b-1804-4b96-88e9-60759c8fd5cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:06.194919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26475a3b-1804-4b96-88e9-60759c8fd5cb service placement] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:06.197373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046717bd-ae38-4dbb-a0b3-4f40fc5c1a15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:06.201872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046717bd-ae38-4dbb-a0b3-4f40fc5c1a15 service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:06.217418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 867/1747] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:06.218200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26475a3b-1804-4b96-88e9-60759c8fd5cb service placement] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:06.232792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26475a3b-1804-4b96-88e9-60759c8fd5cb service placement] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:06.239781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ef5d8-2460-40a7-bb36-f6721db37b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:06.240547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ef5d8-2460-40a7-bb36-f6721db37b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:06.240901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ef5d8-2460-40a7-bb36-f6721db37b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:06.241179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ef5d8-2460-40a7-bb36-f6721db37b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:06.241918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ef5d8-2460-40a7-bb36-f6721db37b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:06.243324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 881/1748] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:06.245141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ef5d8-2460-40a7-bb36-f6721db37b59 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:06.246486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612dca4-5a4a-41a0-8bab-4cff63f76aa6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:06.247102 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612dca4-5a4a-41a0-8bab-4cff63f76aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:06.247910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612dca4-5a4a-41a0-8bab-4cff63f76aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:06.263037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 868/1749] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:06.264017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ef5d8-2460-40a7-bb36-f6721db37b59 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:06.289104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ef5d8-2460-40a7-bb36-f6721db37b59 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:06.317135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-6612dca4-5a4a-41a0-8bab-4cff63f76aa6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:06.318394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496289a-22fb-4187-b0e5-76681cb090e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:06.320824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6612dca4-5a4a-41a0-8bab-4cff63f76aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:06.322143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496289a-22fb-4187-b0e5-76681cb090e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:06.323053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496289a-22fb-4187-b0e5-76681cb090e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:06.323620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496289a-22fb-4187-b0e5-76681cb090e9 admin admin] PATH_INFO: `/v3/projects/d3846e9ebd4c42fa8388f90d8d8ce16e/users/ff3ee318651a490fa8df875bec7de427/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:06.376226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 882/1750] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:06.383477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612dca4-5a4a-41a0-8bab-4cff63f76aa6 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:06.401376 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496289a-22fb-4187-b0e5-76681cb090e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3846e9ebd4c42fa8388f90d8d8ce16e, user_id=ff3ee318651a490fa8df875bec7de427, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:06.405456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612dca4-5a4a-41a0-8bab-4cff63f76aa6 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:06.409649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496289a-22fb-4187-b0e5-76681cb090e9 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:06.415400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70934df-c9d5-4ab2-9316-4b3b6810acac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:06.416570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70934df-c9d5-4ab2-9316-4b3b6810acac admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:06.417252 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70934df-c9d5-4ab2-9316-4b3b6810acac admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:06.417753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70934df-c9d5-4ab2-9316-4b3b6810acac admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:06.419324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70934df-c9d5-4ab2-9316-4b3b6810acac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:06.419985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70934df-c9d5-4ab2-9316-4b3b6810acac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:06.425693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70934df-c9d5-4ab2-9316-4b3b6810acac admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:06.439002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 869/1751] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:06 2022] PUT /identity/v3/projects/d3846e9ebd4c42fa8388f90d8d8ce16e/users/ff3ee318651a490fa8df875bec7de427/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:06.453085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2fcf6-873e-4018-a834-0a5c3c6d58e0 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:06.453381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2fcf6-873e-4018-a834-0a5c3c6d58e0 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:06.453940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2fcf6-873e-4018-a834-0a5c3c6d58e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:06.487391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 883/1752] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:05:06.494539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70934df-c9d5-4ab2-9316-4b3b6810acac admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:06.520349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:06.558434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-dab2fcf6-873e-4018-a834-0a5c3c6d58e0 None None] MFA Rules not processed for user `ff3ee318651a490fa8df875bec7de427`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:06.561884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-dab2fcf6-873e-4018-a834-0a5c3c6d58e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:06.566546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70934df-c9d5-4ab2-9316-4b3b6810acac admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:06.759172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1562bfd-5cb0-4c98-8ba5-f4cd1fa79549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:06.760402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1562bfd-5cb0-4c98-8ba5-f4cd1fa79549 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:06.760857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1562bfd-5cb0-4c98-8ba5-f4cd1fa79549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:06.761245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1562bfd-5cb0-4c98-8ba5-f4cd1fa79549 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:06.762092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1562bfd-5cb0-4c98-8ba5-f4cd1fa79549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:06.764884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1562bfd-5cb0-4c98-8ba5-f4cd1fa79549 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:06.774578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:06.845220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 884/1753] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:06 2022] POST /identity/v3/projects => generated 373 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:06.852012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1562bfd-5cb0-4c98-8ba5-f4cd1fa79549 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:06.853898 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 870/1754] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:06.859048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2fcf6-873e-4018-a834-0a5c3c6d58e0 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:06.878291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2fcf6-873e-4018-a834-0a5c3c6d58e0 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:06.979336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1562bfd-5cb0-4c98-8ba5-f4cd1fa79549 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:06.987850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f07710-afb2-45d0-b91c-3fa8be7022f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:06.987850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f07710-afb2-45d0-b91c-3fa8be7022f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:06.988264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f07710-afb2-45d0-b91c-3fa8be7022f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:06.988648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f07710-afb2-45d0-b91c-3fa8be7022f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:06.989544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f07710-afb2-45d0-b91c-3fa8be7022f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:06.991808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f07710-afb2-45d0-b91c-3fa8be7022f3 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:07.051124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8404c29-2329-42ea-b061-696e0746a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:07.052064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8404c29-2329-42ea-b061-696e0746a656 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:07.052563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8404c29-2329-42ea-b061-696e0746a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:07.053197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8404c29-2329-42ea-b061-696e0746a656 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:07.054079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8404c29-2329-42ea-b061-696e0746a656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:07.056337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8404c29-2329-42ea-b061-696e0746a656 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:07.059432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 885/1755] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:06 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:07.060734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f07710-afb2-45d0-b91c-3fa8be7022f3 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:07.087960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f07710-afb2-45d0-b91c-3fa8be7022f3 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:07.105279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fe9b3-93af-4326-ab36-fdc3bfbc6090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:07.106199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe9b3-93af-4326-ab36-fdc3bfbc6090 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:07.106733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe9b3-93af-4326-ab36-fdc3bfbc6090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:07.107722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe9b3-93af-4326-ab36-fdc3bfbc6090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:07.108870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fe9b3-93af-4326-ab36-fdc3bfbc6090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:07.112523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fe9b3-93af-4326-ab36-fdc3bfbc6090 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:07.138860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 871/1756] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:06 2022] POST /identity/v3/projects => generated 401 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:07.141542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8404c29-2329-42ea-b061-696e0746a656 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:07.168351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 886/1757] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:07.177225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fe9b3-93af-4326-ab36-fdc3bfbc6090 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:07.205570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fe9b3-93af-4326-ab36-fdc3bfbc6090 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:07.485069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e36988-10bf-4417-9efd-0163ca2dfa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:07.489151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e36988-10bf-4417-9efd-0163ca2dfa56 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:07.491296 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e36988-10bf-4417-9efd-0163ca2dfa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:07.494169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e36988-10bf-4417-9efd-0163ca2dfa56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:07.515462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e36988-10bf-4417-9efd-0163ca2dfa56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:07.533223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e36988-10bf-4417-9efd-0163ca2dfa56 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:07.540355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8404c29-2329-42ea-b061-696e0746a656 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:07.584089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836c35-4acb-4f1b-8484-edc99bc859df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:07.586149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836c35-4acb-4f1b-8484-edc99bc859df admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:07.606782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836c35-4acb-4f1b-8484-edc99bc859df admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:07.607989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8836c35-4acb-4f1b-8484-edc99bc859df admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:07.623187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8836c35-4acb-4f1b-8484-edc99bc859df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:07.633330 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8836c35-4acb-4f1b-8484-edc99bc859df admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:07.636753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 887/1758] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:07.647813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e36988-10bf-4417-9efd-0163ca2dfa56 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:07.729169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e36988-10bf-4417-9efd-0163ca2dfa56 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:07.759733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a07d9-5d81-4373-ac66-22b6106d192d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:07.765710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a07d9-5d81-4373-ac66-22b6106d192d admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:07.767555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a07d9-5d81-4373-ac66-22b6106d192d admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:07.768977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a07d9-5d81-4373-ac66-22b6106d192d admin admin] PATH_INFO: `/v3/projects/0e46feba499d403a8000d30b047171d2/users/6af1a995afdc4693a8c89eefa8b3a69c/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:07.793092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 872/1759] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:07 2022] POST /identity/v3/users => generated 307 bytes in 651 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:07.795101 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836c35-4acb-4f1b-8484-edc99bc859df admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:07.844824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8836c35-4acb-4f1b-8484-edc99bc859df admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:07.861857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a07d9-5d81-4373-ac66-22b6106d192d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e46feba499d403a8000d30b047171d2, user_id=6af1a995afdc4693a8c89eefa8b3a69c, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:07.870875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a07d9-5d81-4373-ac66-22b6106d192d admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:07.891791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb54773-7bcc-4247-8eff-693108694662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:07.893631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb54773-7bcc-4247-8eff-693108694662 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:07.893921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb54773-7bcc-4247-8eff-693108694662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:07.898042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb54773-7bcc-4247-8eff-693108694662 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:07.898930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb54773-7bcc-4247-8eff-693108694662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:07.900690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb54773-7bcc-4247-8eff-693108694662 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:07.947011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 888/1760] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:07 2022] PUT /identity/v3/projects/0e46feba499d403a8000d30b047171d2/users/6af1a995afdc4693a8c89eefa8b3a69c/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:07.959991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a07d9-5d81-4373-ac66-22b6106d192d admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:07.962529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 873/1761] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:07.971619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4c85c-e2e4-46a1-bd80-45463dac975a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:07.974625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4c85c-e2e4-46a1-bd80-45463dac975a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:07.974988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4c85c-e2e4-46a1-bd80-45463dac975a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:08.011862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:08.137289 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-bba4c85c-e2e4-46a1-bd80-45463dac975a None None] MFA Rules not processed for user `6af1a995afdc4693a8c89eefa8b3a69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:08.138877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bba4c85c-e2e4-46a1-bd80-45463dac975a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:08.146224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:08.226138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a07d9-5d81-4373-ac66-22b6106d192d admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:08.275251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15f7bb-ac99-426f-b4ae-62c077933aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:08.275989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15f7bb-ac99-426f-b4ae-62c077933aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:08.276274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15f7bb-ac99-426f-b4ae-62c077933aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:08.306492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15f7bb-ac99-426f-b4ae-62c077933aa0 admin admin] PATH_INFO: `/v3/projects/65871cda029148faa8754a06b6b91bbb/users/9545e0af8f644b9992f0db1b14ac2d3d/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:08.430335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15f7bb-ac99-426f-b4ae-62c077933aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65871cda029148faa8754a06b6b91bbb, user_id=9545e0af8f644b9992f0db1b14ac2d3d, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:08.447427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15f7bb-ac99-426f-b4ae-62c077933aa0 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:08.523425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 874/1762] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 560 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:08.534087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4c85c-e2e4-46a1-bd80-45463dac975a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:08.536222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 889/1763] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:07 2022] PUT /identity/v3/projects/65871cda029148faa8754a06b6b91bbb/users/9545e0af8f644b9992f0db1b14ac2d3d/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:08.541344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15f7bb-ac99-426f-b4ae-62c077933aa0 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:08.555853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4c85c-e2e4-46a1-bd80-45463dac975a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:08.862619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15f7bb-ac99-426f-b4ae-62c077933aa0 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:08.891891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f9805-0c35-4ff4-906d-c56a009231ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:08.892767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f9805-0c35-4ff4-906d-c56a009231ba admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:08.897893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f9805-0c35-4ff4-906d-c56a009231ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:08.898488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f9805-0c35-4ff4-906d-c56a009231ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:08.916053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f9805-0c35-4ff4-906d-c56a009231ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:08.917811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f9805-0c35-4ff4-906d-c56a009231ba admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:08.921224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431c5cb-698a-4e34-b9e5-7f3c4a85d12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:08.922080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431c5cb-698a-4e34-b9e5-7f3c4a85d12b admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:08.922504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431c5cb-698a-4e34-b9e5-7f3c4a85d12b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:08.923124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431c5cb-698a-4e34-b9e5-7f3c4a85d12b admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:08.924090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431c5cb-698a-4e34-b9e5-7f3c4a85d12b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:08.926842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431c5cb-698a-4e34-b9e5-7f3c4a85d12b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:08.979486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 890/1764] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:09.000120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dc037-e445-4209-a320-e8650de8be1f None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:09.000577 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dc037-e445-4209-a320-e8650de8be1f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:09.000813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5dc037-e445-4209-a320-e8650de8be1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:09.010052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 875/1765] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:08 2022] POST /identity/v3/projects => generated 371 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:09.011294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431c5cb-698a-4e34-b9e5-7f3c4a85d12b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:09.069505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1d5dc037-e445-4209-a320-e8650de8be1f None None] MFA Rules not processed for user `fc8a5fa04b6141329778221169afdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:09.073276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1d5dc037-e445-4209-a320-e8650de8be1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:09.274020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431c5cb-698a-4e34-b9e5-7f3c4a85d12b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:09.293471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e204f3f-3f40-480a-b5dc-b74c1a9ebc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:09.294358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e204f3f-3f40-480a-b5dc-b74c1a9ebc6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:09.294849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e204f3f-3f40-480a-b5dc-b74c1a9ebc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:09.296817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e204f3f-3f40-480a-b5dc-b74c1a9ebc6a admin admin] PATH_INFO: `/v3/projects/65871cda029148faa8754a06b6b91bbb/users/9545e0af8f644b9992f0db1b14ac2d3d/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:09.303344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e204f3f-3f40-480a-b5dc-b74c1a9ebc6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=65871cda029148faa8754a06b6b91bbb, user_id=9545e0af8f644b9992f0db1b14ac2d3d, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:09.313843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e204f3f-3f40-480a-b5dc-b74c1a9ebc6a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:09.354315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 891/1766] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:09.356653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5dc037-e445-4209-a320-e8650de8be1f None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:09.374095 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 876/1767] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:09 2022] PUT /identity/v3/projects/65871cda029148faa8754a06b6b91bbb/users/9545e0af8f644b9992f0db1b14ac2d3d/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:09.389632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5dc037-e445-4209-a320-e8650de8be1f None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:09.391711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5786b37-91a8-4ad1-bcb2-3df3154ded69 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:09.398180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5786b37-91a8-4ad1-bcb2-3df3154ded69 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:09.399780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5786b37-91a8-4ad1-bcb2-3df3154ded69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:09.510913 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:09.533852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d5786b37-91a8-4ad1-bcb2-3df3154ded69 None None] MFA Rules not processed for user `9545e0af8f644b9992f0db1b14ac2d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:09.537038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d5786b37-91a8-4ad1-bcb2-3df3154ded69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:09.624037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e82e-7cf5-4655-b869-3fc0aec0bba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:09.624740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e82e-7cf5-4655-b869-3fc0aec0bba8 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:09.625022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e82e-7cf5-4655-b869-3fc0aec0bba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:09.625299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8e82e-7cf5-4655-b869-3fc0aec0bba8 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:09.626008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e82e-7cf5-4655-b869-3fc0aec0bba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:09.630555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e8e82e-7cf5-4655-b869-3fc0aec0bba8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:09.666242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:09.718761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 892/1768] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:09 2022] POST /identity/v3/users => generated 292 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:09.721280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e82e-7cf5-4655-b869-3fc0aec0bba8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:09.759467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e8e82e-7cf5-4655-b869-3fc0aec0bba8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:09.783536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e4cf6-9d1d-4ce3-9509-93badf0a7da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:09.784275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e4cf6-9d1d-4ce3-9509-93badf0a7da9 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:09.784654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e4cf6-9d1d-4ce3-9509-93badf0a7da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:09.784985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e4cf6-9d1d-4ce3-9509-93badf0a7da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:09.786453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e4cf6-9d1d-4ce3-9509-93badf0a7da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:09.788579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e4cf6-9d1d-4ce3-9509-93badf0a7da9 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:09.798918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 877/1769] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:09 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:09.898542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 893/1770] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:09.899843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5786b37-91a8-4ad1-bcb2-3df3154ded69 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:09.914083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e4cf6-9d1d-4ce3-9509-93badf0a7da9 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:09.924191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3d7e4cf6-9d1d-4ce3-9509-93badf0a7da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:09.931759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5786b37-91a8-4ad1-bcb2-3df3154ded69 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:09.965523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c20cb-9b80-4299-9664-3d208277a2ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:09.970859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c20cb-9b80-4299-9664-3d208277a2ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:09.971589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c20cb-9b80-4299-9664-3d208277a2ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:09.971871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c20cb-9b80-4299-9664-3d208277a2ed admin admin] PATH_INFO: `/v3/projects/61e653530b5b4ed6a121c401ed2f3895/users/f8e3f761fc3b42668fe412dbc649f7e6/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:10.040156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c20cb-9b80-4299-9664-3d208277a2ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e653530b5b4ed6a121c401ed2f3895, user_id=f8e3f761fc3b42668fe412dbc649f7e6, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:10.053656 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c20cb-9b80-4299-9664-3d208277a2ed admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:10.095196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 878/1771] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:09 2022] PUT /identity/v3/projects/61e653530b5b4ed6a121c401ed2f3895/users/f8e3f761fc3b42668fe412dbc649f7e6/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:10.099154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c20cb-9b80-4299-9664-3d208277a2ed admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:10.121198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e4cf6-9d1d-4ce3-9509-93badf0a7da9 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:10.232694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c20cb-9b80-4299-9664-3d208277a2ed admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:10.242049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79645b15-721c-462b-80bc-db6089fe9aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:10.242924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79645b15-721c-462b-80bc-db6089fe9aec admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:10.243956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79645b15-721c-462b-80bc-db6089fe9aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:10.244366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79645b15-721c-462b-80bc-db6089fe9aec admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:10.246055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79645b15-721c-462b-80bc-db6089fe9aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:10.247251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79645b15-721c-462b-80bc-db6089fe9aec admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:10.268962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 879/1772] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:10.274622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79645b15-721c-462b-80bc-db6089fe9aec admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:10.298199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79645b15-721c-462b-80bc-db6089fe9aec admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:10.307005 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3d03cb-8351-4291-adc9-bacdd52723eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:10.308197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d03cb-8351-4291-adc9-bacdd52723eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:10.308349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d03cb-8351-4291-adc9-bacdd52723eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:10.308611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3d03cb-8351-4291-adc9-bacdd52723eb admin admin] PATH_INFO: `/v3/projects/61e653530b5b4ed6a121c401ed2f3895/users/f8e3f761fc3b42668fe412dbc649f7e6/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:10.312752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d03cb-8351-4291-adc9-bacdd52723eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e653530b5b4ed6a121c401ed2f3895, user_id=f8e3f761fc3b42668fe412dbc649f7e6, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:10.316353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3d03cb-8351-4291-adc9-bacdd52723eb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:10.337600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889abb4-31e2-4b81-9b4d-9735f80dce29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:10.338629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889abb4-31e2-4b81-9b4d-9735f80dce29 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:10.339076 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889abb4-31e2-4b81-9b4d-9735f80dce29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:10.339634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889abb4-31e2-4b81-9b4d-9735f80dce29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:10.355588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 880/1773] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:10 2022] PUT /identity/v3/projects/61e653530b5b4ed6a121c401ed2f3895/users/f8e3f761fc3b42668fe412dbc649f7e6/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:10.357707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661510f-1e20-4ae2-a51e-7c6a86f7e539 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:10.358004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661510f-1e20-4ae2-a51e-7c6a86f7e539 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:10.358287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661510f-1e20-4ae2-a51e-7c6a86f7e539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:10.400872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c661510f-1e20-4ae2-a51e-7c6a86f7e539 None None] MFA Rules not processed for user `f8e3f761fc3b42668fe412dbc649f7e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:10.402591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c661510f-1e20-4ae2-a51e-7c6a86f7e539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:10.514246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889abb4-31e2-4b81-9b4d-9735f80dce29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:10.515780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889abb4-31e2-4b81-9b4d-9735f80dce29 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:10.525846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:10.558253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 894/1774] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:09 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:10.636182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 881/1775] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:10 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:10.679171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889abb4-31e2-4b81-9b4d-9735f80dce29 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:10.694607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889abb4-31e2-4b81-9b4d-9735f80dce29 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:10.803577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661510f-1e20-4ae2-a51e-7c6a86f7e539 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:10.808321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c661510f-1e20-4ae2-a51e-7c6a86f7e539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:10.835642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:10.862359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865ad50-5d2c-434b-9dfe-76e1efdda92f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:10.863009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865ad50-5d2c-434b-9dfe-76e1efdda92f service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:10.863318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865ad50-5d2c-434b-9dfe-76e1efdda92f service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:10.863572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865ad50-5d2c-434b-9dfe-76e1efdda92f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:10.889050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865ad50-5d2c-434b-9dfe-76e1efdda92f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:10.890582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865ad50-5d2c-434b-9dfe-76e1efdda92f service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:10.922489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 895/1776] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:10 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:11.047337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661510f-1e20-4ae2-a51e-7c6a86f7e539 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:11.085481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96d497-02c0-4818-8e3b-8bcbed15f1d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:11.086259 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96d497-02c0-4818-8e3b-8bcbed15f1d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:11.086520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96d497-02c0-4818-8e3b-8bcbed15f1d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:11.087084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96d497-02c0-4818-8e3b-8bcbed15f1d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:11.339729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96d497-02c0-4818-8e3b-8bcbed15f1d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:11.341447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96d497-02c0-4818-8e3b-8bcbed15f1d8 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:11.367496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 882/1777] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:12.189215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e187a-9e70-4d62-9fe9-9c369c8452fe None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:12.190330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e187a-9e70-4d62-9fe9-9c369c8452fe None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:12.190940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e187a-9e70-4d62-9fe9-9c369c8452fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:12.402138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3b8e187a-9e70-4d62-9fe9-9c369c8452fe None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:12.406117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3b8e187a-9e70-4d62-9fe9-9c369c8452fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:12.836736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 896/1778] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:05:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 647 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:13.121627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96d497-02c0-4818-8e3b-8bcbed15f1d8 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:13.123856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ff96d497-02c0-4818-8e3b-8bcbed15f1d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:13.449148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96d497-02c0-4818-8e3b-8bcbed15f1d8 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:13.464256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de980538-65dd-4799-89a8-7281320b4d82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:13.466144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de980538-65dd-4799-89a8-7281320b4d82 service swift] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:13.468026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de980538-65dd-4799-89a8-7281320b4d82 service swift] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:13.469136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de980538-65dd-4799-89a8-7281320b4d82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:13.515178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de980538-65dd-4799-89a8-7281320b4d82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:13.521252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de980538-65dd-4799-89a8-7281320b4d82 service swift] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:13.559633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 883/1779] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:05:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:05:14.063197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593597-5b43-491a-86f5-858bf3b3737e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:14.063708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593597-5b43-491a-86f5-858bf3b3737e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:14.076719 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593597-5b43-491a-86f5-858bf3b3737e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:14.103289 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de980538-65dd-4799-89a8-7281320b4d82 service swift] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:14.111096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:14.245961 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-69593597-5b43-491a-86f5-858bf3b3737e None None] MFA Rules not processed for user `e98a16b117294e10b9d0c8c721d8e114`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:14.247746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-69593597-5b43-491a-86f5-858bf3b3737e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:14.564117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de980538-65dd-4799-89a8-7281320b4d82 service swift] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:14.598986 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:14.602392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:14.602678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:14.602958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] PATH_INFO: `/v3/users/6af1a995afdc4693a8c89eefa8b3a69c` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:14.610784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6af1a995afdc4693a8c89eefa8b3a69c)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:14.630236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:14.747552 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 897/1780] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 685 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:14.891263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6af1a995afdc4693a8c89eefa8b3a69c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:14.892254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6af1a995afdc4693a8c89eefa8b3a69c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:14.915095 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6af1a995afdc4693a8c89eefa8b3a69c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:14.974930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6af1a995afdc4693a8c89eefa8b3a69c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:15.003659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6af1a995afdc4693a8c89eefa8b3a69c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:15.041769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 884/1781] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:14 2022] DELETE /identity/v3/users/6af1a995afdc4693a8c89eefa8b3a69c => generated 0 bytes in 939 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:15.047534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4f27c-60b4-4334-acdc-7ea97675c10d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:15.047911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4f27c-60b4-4334-acdc-7ea97675c10d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:15.049339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4f27c-60b4-4334-acdc-7ea97675c10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:15.117821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-fca4f27c-60b4-4334-acdc-7ea97675c10d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:15.119467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fca4f27c-60b4-4334-acdc-7ea97675c10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:15.326679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 898/1782] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:15.406851 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:15.408463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:15.631966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c58e2-e117-489a-864b-bf67a0e2e90b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:15.642443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e2ef-49a4-40e3-a0e6-a7aa3aadacbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:15.642820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e2ef-49a4-40e3-a0e6-a7aa3aadacbc service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:15.643099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e2ef-49a4-40e3-a0e6-a7aa3aadacbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:15.643373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849e2ef-49a4-40e3-a0e6-a7aa3aadacbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:15.662892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849e2ef-49a4-40e3-a0e6-a7aa3aadacbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:15.665030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849e2ef-49a4-40e3-a0e6-a7aa3aadacbc service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:15.749914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 885/1783] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:17.525056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4f27c-60b4-4334-acdc-7ea97675c10d None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:17.527799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fca4f27c-60b4-4334-acdc-7ea97675c10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:17.545732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:17.556989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4f27c-60b4-4334-acdc-7ea97675c10d None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:17.584328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:17.586770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:17.589658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:17.590156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] PATH_INFO: `/v3/projects/0e46feba499d403a8000d30b047171d2` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:17.601250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e46feba499d403a8000d30b047171d2)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:17.611000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:17.745932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e46feba499d403a8000d30b047171d2', 'request_id': 'req-f1590f5f-a1ac-4925-9c13-c4d67836813b'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:17.757735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e46feba499d403a8000d30b047171d2', 'request_id': 'req-f1590f5f-a1ac-4925-9c13-c4d67836813b'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:17.788045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e46feba499d403a8000d30b047171d2', 'request_id': 'req-f1590f5f-a1ac-4925-9c13-c4d67836813b'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:17.883128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] The token cache is being invalidate because project 0e46feba499d403a8000d30b047171d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:17.884608 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:17.891100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:17.901134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e2ef-49a4-40e3-a0e6-a7aa3aadacbc service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:17.901658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 899/1784] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:17 2022] DELETE /identity/v3/projects/0e46feba499d403a8000d30b047171d2 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:17.916724 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849e2ef-49a4-40e3-a0e6-a7aa3aadacbc service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:18.036585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:18.136566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:18.137496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:18.137967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:18.138396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] PATH_INFO: `/v3/users/f8e3f761fc3b42668fe412dbc649f7e6` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:18.139803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8e3f761fc3b42668fe412dbc649f7e6)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:18.142323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:18.224936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8e3f761fc3b42668fe412dbc649f7e6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:18.225867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8e3f761fc3b42668fe412dbc649f7e6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:18.242295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8e3f761fc3b42668fe412dbc649f7e6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:18.274283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8e3f761fc3b42668fe412dbc649f7e6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:18.288691 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e02063e6-631b-41b5-ba41-91d92f965e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8e3f761fc3b42668fe412dbc649f7e6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:18.302360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 886/1785] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:17 2022] DELETE /identity/v3/users/f8e3f761fc3b42668fe412dbc649f7e6 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:18.726726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:18.729447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:18.744310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1590f5f-a1ac-4925-9c13-c4d67836813b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:18.852353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:18.853205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:18.853488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:18.853767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] PATH_INFO: `/v3/projects/61e653530b5b4ed6a121c401ed2f3895` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:18.855101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61e653530b5b4ed6a121c401ed2f3895)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:18.857102 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:18.935017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61e653530b5b4ed6a121c401ed2f3895', 'request_id': 'req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:18.941656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61e653530b5b4ed6a121c401ed2f3895', 'request_id': 'req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:18.950215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61e653530b5b4ed6a121c401ed2f3895', 'request_id': 'req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:18.961707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] The token cache is being invalidate because project 61e653530b5b4ed6a121c401ed2f3895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:18.961998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:18.962645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3ecb9577-875c-4c44-bf9f-7ff8ee1ecf65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:18.964192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 900/1786] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:18 2022] DELETE /identity/v3/projects/61e653530b5b4ed6a121c401ed2f3895 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:18.969154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b354a0-d065-44cf-b0b5-717c1acb63ec None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:18.969681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b354a0-d065-44cf-b0b5-717c1acb63ec None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:18.969806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b354a0-d065-44cf-b0b5-717c1acb63ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:18.990908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-35b354a0-d065-44cf-b0b5-717c1acb63ec None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:18.992116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-35b354a0-d065-44cf-b0b5-717c1acb63ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:19.061821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 887/1787] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:19.069594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fdcb4-c0f2-4ea5-887d-55bf41cd2eb4 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.069879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fdcb4-c0f2-4ea5-887d-55bf41cd2eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.070207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fdcb4-c0f2-4ea5-887d-55bf41cd2eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.090182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-244fdcb4-c0f2-4ea5-887d-55bf41cd2eb4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:19.091413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-244fdcb4-c0f2-4ea5-887d-55bf41cd2eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:19.104623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 901/1788] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:19.110252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b354a0-d065-44cf-b0b5-717c1acb63ec None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.121090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b354a0-d065-44cf-b0b5-717c1acb63ec None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:19.126583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fa363-ab6c-4025-8c94-5fc27ea51f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:19.127227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fa363-ab6c-4025-8c94-5fc27ea51f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.127489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fa363-ab6c-4025-8c94-5fc27ea51f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.127761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27fa363-ab6c-4025-8c94-5fc27ea51f6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.128493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fa363-ab6c-4025-8c94-5fc27ea51f6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:19.128901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fa363-ab6c-4025-8c94-5fc27ea51f6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:19.130529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27fa363-ab6c-4025-8c94-5fc27ea51f6d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:19.142987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 888/1789] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:05:19.147640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fdcb4-c0f2-4ea5-887d-55bf41cd2eb4 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.157334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fdcb4-c0f2-4ea5-887d-55bf41cd2eb4 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:19.162351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c687b04-913e-4c27-82bb-4b0abad453d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:19.162973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c687b04-913e-4c27-82bb-4b0abad453d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.163253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c687b04-913e-4c27-82bb-4b0abad453d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.163525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c687b04-913e-4c27-82bb-4b0abad453d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.164197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c687b04-913e-4c27-82bb-4b0abad453d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:19.166145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c687b04-913e-4c27-82bb-4b0abad453d8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:19.176442 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:19.200794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 902/1790] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:19 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:19.205137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fa363-ab6c-4025-8c94-5fc27ea51f6d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.218820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27fa363-ab6c-4025-8c94-5fc27ea51f6d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:19.255365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:19.286327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705785c-8313-4adb-8e74-0b2ae3bfc995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:19.287004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705785c-8313-4adb-8e74-0b2ae3bfc995 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.287284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705785c-8313-4adb-8e74-0b2ae3bfc995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.287556 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705785c-8313-4adb-8e74-0b2ae3bfc995 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.288246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705785c-8313-4adb-8e74-0b2ae3bfc995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:19.290218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705785c-8313-4adb-8e74-0b2ae3bfc995 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:19.324639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 889/1791] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:19 2022] POST /identity/v3/users => generated 282 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:19.329574 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c687b04-913e-4c27-82bb-4b0abad453d8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.341624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c687b04-913e-4c27-82bb-4b0abad453d8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:19.347525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c819bc4-34d9-48c7-8758-b478dd69d33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:19.348235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819bc4-34d9-48c7-8758-b478dd69d33f admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.348547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819bc4-34d9-48c7-8758-b478dd69d33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.348888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819bc4-34d9-48c7-8758-b478dd69d33f admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.349562 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c819bc4-34d9-48c7-8758-b478dd69d33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:19.350132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133ef19-1862-426a-bfb7-ad2d7ba177d7 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.350418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133ef19-1862-426a-bfb7-ad2d7ba177d7 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.350695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133ef19-1862-426a-bfb7-ad2d7ba177d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.351251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c819bc4-34d9-48c7-8758-b478dd69d33f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:19.364784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 903/1792] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:19.369966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c819bc4-34d9-48c7-8758-b478dd69d33f admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.375548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-1133ef19-1862-426a-bfb7-ad2d7ba177d7 None None] MFA Rules not processed for user `8dfbda2838b242558c885f913fafb5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:19.377060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1133ef19-1862-426a-bfb7-ad2d7ba177d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:19.385392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c819bc4-34d9-48c7-8758-b478dd69d33f admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:19.391726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bd981-89e7-421f-b2d8-4ea915d5fdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:19.392624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bd981-89e7-421f-b2d8-4ea915d5fdc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.392915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bd981-89e7-421f-b2d8-4ea915d5fdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.393186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bd981-89e7-421f-b2d8-4ea915d5fdc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.393977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bd981-89e7-421f-b2d8-4ea915d5fdc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:19.395701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4bd981-89e7-421f-b2d8-4ea915d5fdc5 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:19.418400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 904/1793] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:05:19.420420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bd981-89e7-421f-b2d8-4ea915d5fdc5 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.431521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4bd981-89e7-421f-b2d8-4ea915d5fdc5 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:19.436607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce625a-864c-45c8-8c2d-e0637b36a9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:19.437315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce625a-864c-45c8-8c2d-e0637b36a9b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.437602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce625a-864c-45c8-8c2d-e0637b36a9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.438011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce625a-864c-45c8-8c2d-e0637b36a9b3 admin admin] PATH_INFO: `/v3/projects/00172503ace840e0944bdd3502c0adc2/users/d574c9eff2f740e0a1578ae3f80831db/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.457257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 890/1794] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Aug 1 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:19.460999 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce625a-864c-45c8-8c2d-e0637b36a9b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00172503ace840e0944bdd3502c0adc2, user_id=d574c9eff2f740e0a1578ae3f80831db, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:19.466393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce625a-864c-45c8-8c2d-e0637b36a9b3 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:19.484417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 905/1795] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:19 2022] PUT /identity/v3/projects/00172503ace840e0944bdd3502c0adc2/users/d574c9eff2f740e0a1578ae3f80831db/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:19.499034 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80facd-a77a-4d29-bf0a-9b9a5609d9ee None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.499494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80facd-a77a-4d29-bf0a-9b9a5609d9ee None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.499862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80facd-a77a-4d29-bf0a-9b9a5609d9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.531246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4e80facd-a77a-4d29-bf0a-9b9a5609d9ee None None] MFA Rules not processed for user `d574c9eff2f740e0a1578ae3f80831db`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:19.532554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4e80facd-a77a-4d29-bf0a-9b9a5609d9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:19.613292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 891/1796] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:19 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:19.617845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce625a-864c-45c8-8c2d-e0637b36a9b3 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.688574 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce625a-864c-45c8-8c2d-e0637b36a9b3 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:19.693913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639a9e7-6825-48ee-9314-05ca284034a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:19.694529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639a9e7-6825-48ee-9314-05ca284034a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.694821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639a9e7-6825-48ee-9314-05ca284034a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.695091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0639a9e7-6825-48ee-9314-05ca284034a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.695783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639a9e7-6825-48ee-9314-05ca284034a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:19.697756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0639a9e7-6825-48ee-9314-05ca284034a7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:19.728904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 906/1797] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:19 2022] POST /identity/v3/projects => generated 363 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:19.733097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80facd-a77a-4d29-bf0a-9b9a5609d9ee None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.745005 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80facd-a77a-4d29-bf0a-9b9a5609d9ee None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:19.822784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00ff8c-bd22-4ce2-9f49-b5172afecbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:19.823493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00ff8c-bd22-4ce2-9f49-b5172afecbdb admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.823748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00ff8c-bd22-4ce2-9f49-b5172afecbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.824032 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00ff8c-bd22-4ce2-9f49-b5172afecbdb admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.824849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00ff8c-bd22-4ce2-9f49-b5172afecbdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:19.826920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00ff8c-bd22-4ce2-9f49-b5172afecbdb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:19.837033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:19.854603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 892/1798] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:19 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:19.859091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639a9e7-6825-48ee-9314-05ca284034a7 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.872159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0639a9e7-6825-48ee-9314-05ca284034a7 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:19.878172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a6895-4257-4d63-848e-83953e59c9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:19.878831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a6895-4257-4d63-848e-83953e59c9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.879366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a6895-4257-4d63-848e-83953e59c9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.879869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a6895-4257-4d63-848e-83953e59c9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.880771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a6895-4257-4d63-848e-83953e59c9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:19.882569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a6895-4257-4d63-848e-83953e59c9a6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:19.895216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 907/1799] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:19.899868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00ff8c-bd22-4ce2-9f49-b5172afecbdb admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.913173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00ff8c-bd22-4ce2-9f49-b5172afecbdb admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:19.918502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab42f3-67a8-446c-b8ad-d12ef0757800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:19.919201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab42f3-67a8-446c-b8ad-d12ef0757800 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:19.919483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab42f3-67a8-446c-b8ad-d12ef0757800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:19.919756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab42f3-67a8-446c-b8ad-d12ef0757800 admin admin] PATH_INFO: `/v3/projects/ba9d2afc92c147b3970004956c9ffae2/users/47ac37a64dce4131927de3a3f8205fe6/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:19.938626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab42f3-67a8-446c-b8ad-d12ef0757800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9d2afc92c147b3970004956c9ffae2, user_id=47ac37a64dce4131927de3a3f8205fe6, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:19.941644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab42f3-67a8-446c-b8ad-d12ef0757800 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:19.957136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 893/1800] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:19 2022] PUT /identity/v3/projects/ba9d2afc92c147b3970004956c9ffae2/users/47ac37a64dce4131927de3a3f8205fe6/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:19.961350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a6895-4257-4d63-848e-83953e59c9a6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:19.972090 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a6895-4257-4d63-848e-83953e59c9a6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:20.035980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03576cd-7b1c-4a9f-966f-839b0249c23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:20.036657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03576cd-7b1c-4a9f-966f-839b0249c23c admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:20.036944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03576cd-7b1c-4a9f-966f-839b0249c23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:20.037217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03576cd-7b1c-4a9f-966f-839b0249c23c admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:20.037922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03576cd-7b1c-4a9f-966f-839b0249c23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:20.039570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03576cd-7b1c-4a9f-966f-839b0249c23c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:20.049622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 908/1801] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:20.053994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab42f3-67a8-446c-b8ad-d12ef0757800 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:20.066442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab42f3-67a8-446c-b8ad-d12ef0757800 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:20.072436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5945be-02e3-47ab-8af5-035730118acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:20.073157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5945be-02e3-47ab-8af5-035730118acb admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:20.073422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5945be-02e3-47ab-8af5-035730118acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:20.073793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5945be-02e3-47ab-8af5-035730118acb admin admin] PATH_INFO: `/v3/projects/ba9d2afc92c147b3970004956c9ffae2/users/47ac37a64dce4131927de3a3f8205fe6/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:20.075781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5945be-02e3-47ab-8af5-035730118acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9d2afc92c147b3970004956c9ffae2, user_id=47ac37a64dce4131927de3a3f8205fe6, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:20.078490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5945be-02e3-47ab-8af5-035730118acb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:20.089908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 894/1802] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:20 2022] PUT /identity/v3/projects/ba9d2afc92c147b3970004956c9ffae2/users/47ac37a64dce4131927de3a3f8205fe6/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:20.101492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54793d-982f-42e2-9cff-4723ac3a491e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:20.101779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54793d-982f-42e2-9cff-4723ac3a491e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:20.102069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54793d-982f-42e2-9cff-4723ac3a491e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:20.130292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:20.133786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-2c54793d-982f-42e2-9cff-4723ac3a491e None None] MFA Rules not processed for user `47ac37a64dce4131927de3a3f8205fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:20.135654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2c54793d-982f-42e2-9cff-4723ac3a491e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:20.208443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 909/1803] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:20 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:20.222535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5945be-02e3-47ab-8af5-035730118acb admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:20.224385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4b5945be-02e3-47ab-8af5-035730118acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:20.284030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5945be-02e3-47ab-8af5-035730118acb admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:20.295268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fce2d5-0f19-4318-a34f-3a2f0c1e170b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:20.295923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fce2d5-0f19-4318-a34f-3a2f0c1e170b service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:20.296325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fce2d5-0f19-4318-a34f-3a2f0c1e170b service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:20.296609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fce2d5-0f19-4318-a34f-3a2f0c1e170b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:20.307727 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fce2d5-0f19-4318-a34f-3a2f0c1e170b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:20.309544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fce2d5-0f19-4318-a34f-3a2f0c1e170b service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:20.332338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 895/1804] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:20 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:20.899341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54793d-982f-42e2-9cff-4723ac3a491e None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:20.995294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54793d-982f-42e2-9cff-4723ac3a491e None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:21.007752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:21.008472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:21.008755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:21.009024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] PATH_INFO: `/v3/users/d574c9eff2f740e0a1578ae3f80831db` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:21.011142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d574c9eff2f740e0a1578ae3f80831db)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:21.017334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:21.125879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd574c9eff2f740e0a1578ae3f80831db'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:21.142374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd574c9eff2f740e0a1578ae3f80831db'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:21.152169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd574c9eff2f740e0a1578ae3f80831db'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:21.153133 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd574c9eff2f740e0a1578ae3f80831db'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:21.153925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd574c9eff2f740e0a1578ae3f80831db'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:21.181035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 910/1805] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:20 2022] DELETE /identity/v3/users/d574c9eff2f740e0a1578ae3f80831db => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:21.186674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7b62d-ec53-442b-b4dc-b09e72db49f8 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:21.187257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7b62d-ec53-442b-b4dc-b09e72db49f8 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:21.187257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7b62d-ec53-442b-b4dc-b09e72db49f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:21.230636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-71b7b62d-ec53-442b-b4dc-b09e72db49f8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:21.231909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-71b7b62d-ec53-442b-b4dc-b09e72db49f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:21.379200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 896/1806] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:21.392261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:21.395421 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:21.452004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:21.559578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b91f0-950a-4684-830b-ff137eead1a6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:21.610352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5f6c5-8d97-41db-b52b-a6e5698b2009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:21.612330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5f6c5-8d97-41db-b52b-a6e5698b2009 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:21.612733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5f6c5-8d97-41db-b52b-a6e5698b2009 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:21.614432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5f6c5-8d97-41db-b52b-a6e5698b2009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:21.647971 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5f6c5-8d97-41db-b52b-a6e5698b2009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:21.652408 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5f6c5-8d97-41db-b52b-a6e5698b2009 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:21.696751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 911/1807] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:21.939898 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7b62d-ec53-442b-b4dc-b09e72db49f8 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:21.941057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5f6c5-8d97-41db-b52b-a6e5698b2009 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:21.942818 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-71b7b62d-ec53-442b-b4dc-b09e72db49f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:21.943298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8dc5f6c5-8d97-41db-b52b-a6e5698b2009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:21.963077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7b62d-ec53-442b-b4dc-b09e72db49f8 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:21.968725 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5f6c5-8d97-41db-b52b-a6e5698b2009 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:21.969420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60826678-913b-4cd4-8117-163d77514213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:21.970232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826678-913b-4cd4-8117-163d77514213 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:21.970668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826678-913b-4cd4-8117-163d77514213 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:21.971140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826678-913b-4cd4-8117-163d77514213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:21.976880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-60826678-913b-4cd4-8117-163d77514213 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:22.001839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60826678-913b-4cd4-8117-163d77514213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:22.004012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60826678-913b-4cd4-8117-163d77514213 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:22.018444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a4ba6-7287-4f9d-a69b-70f3396f38a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:22.019550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4ba6-7287-4f9d-a69b-70f3396f38a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:22.025934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 897/1808] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:22.031009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60826678-913b-4cd4-8117-163d77514213 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:22.034728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4ba6-7287-4f9d-a69b-70f3396f38a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:22.037934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a4ba6-7287-4f9d-a69b-70f3396f38a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:22.053709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60826678-913b-4cd4-8117-163d77514213 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:22.067965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa43b3a-aa8f-437e-b420-8eef63847b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:22.069215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa43b3a-aa8f-437e-b420-8eef63847b39 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:22.069508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa43b3a-aa8f-437e-b420-8eef63847b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:22.069786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa43b3a-aa8f-437e-b420-8eef63847b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:22.089169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa43b3a-aa8f-437e-b420-8eef63847b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:22.095160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa43b3a-aa8f-437e-b420-8eef63847b39 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:22.118005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4ba6-7287-4f9d-a69b-70f3396f38a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:22.131854 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a4ba6-7287-4f9d-a69b-70f3396f38a9 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:22.137670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 898/1809] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:22.139776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa43b3a-aa8f-437e-b420-8eef63847b39 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:22.190580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa43b3a-aa8f-437e-b420-8eef63847b39 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:22.203276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904edda1-cf4e-49f7-b48b-79ff70a33fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:22.204327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904edda1-cf4e-49f7-b48b-79ff70a33fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:22.204792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904edda1-cf4e-49f7-b48b-79ff70a33fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:22.206052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904edda1-cf4e-49f7-b48b-79ff70a33fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:22.241818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 912/1810] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:22.261679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904edda1-cf4e-49f7-b48b-79ff70a33fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:22.264325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904edda1-cf4e-49f7-b48b-79ff70a33fb8 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:22.322297 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 899/1811] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:22.423086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a4ba6-7287-4f9d-a69b-70f3396f38a9 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:22.425419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-967a4ba6-7287-4f9d-a69b-70f3396f38a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:22.462729 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a4ba6-7287-4f9d-a69b-70f3396f38a9 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:22.478795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:22.480709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:22.481000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:22.482617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] PATH_INFO: `/v3/users/ff3ee318651a490fa8df875bec7de427` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:22.486649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff3ee318651a490fa8df875bec7de427)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:22.494533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:22.718056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff3ee318651a490fa8df875bec7de427'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:22.742129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff3ee318651a490fa8df875bec7de427'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:22.758824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff3ee318651a490fa8df875bec7de427'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:22.761105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff3ee318651a490fa8df875bec7de427'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:22.761703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff3ee318651a490fa8df875bec7de427'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:22.836482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 913/1812] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:22 2022] DELETE /identity/v3/users/ff3ee318651a490fa8df875bec7de427 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:22.841657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06092e7-70fe-42af-be5a-d2311e49d1e2 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:22.841984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06092e7-70fe-42af-be5a-d2311e49d1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:22.842265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06092e7-70fe-42af-be5a-d2311e49d1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:22.901806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c06092e7-70fe-42af-be5a-d2311e49d1e2 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:22.903043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c06092e7-70fe-42af-be5a-d2311e49d1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:22.952130 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:23.081281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 900/1813] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:23.186808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:23.189678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:23.433829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f72f8-3be9-4816-8125-fb030b59570b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:23.455734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812bca5-d906-42d3-adb7-cb1e98017d83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:23.455734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812bca5-d906-42d3-adb7-cb1e98017d83 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:23.455734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812bca5-d906-42d3-adb7-cb1e98017d83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:23.455734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812bca5-d906-42d3-adb7-cb1e98017d83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:23.480166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812bca5-d906-42d3-adb7-cb1e98017d83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:23.483311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812bca5-d906-42d3-adb7-cb1e98017d83 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:23.504060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 914/1814] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:23.722826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06092e7-70fe-42af-be5a-d2311e49d1e2 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:23.724447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c06092e7-70fe-42af-be5a-d2311e49d1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:23.750403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06092e7-70fe-42af-be5a-d2311e49d1e2 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:23.755967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:23.756590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:23.756856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:23.757347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] PATH_INFO: `/v3/projects/00172503ace840e0944bdd3502c0adc2` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:23.761213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] RBAC: Authorizing `identity:delete_project(project_id=00172503ace840e0944bdd3502c0adc2)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:23.763501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:23.875569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00172503ace840e0944bdd3502c0adc2', 'request_id': 'req-f8fcca34-b1be-44ae-97ec-304174cc246c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:23.898212 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00172503ace840e0944bdd3502c0adc2', 'request_id': 'req-f8fcca34-b1be-44ae-97ec-304174cc246c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:23.911216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00172503ace840e0944bdd3502c0adc2', 'request_id': 'req-f8fcca34-b1be-44ae-97ec-304174cc246c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:23.929984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] The token cache is being invalidate because project 00172503ace840e0944bdd3502c0adc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:23.930285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:23.931593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8fcca34-b1be-44ae-97ec-304174cc246c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:23.938173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 901/1815] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:23 2022] DELETE /identity/v3/projects/00172503ace840e0944bdd3502c0adc2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:23.942651 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812bca5-d906-42d3-adb7-cb1e98017d83 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:23.957258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812bca5-d906-42d3-adb7-cb1e98017d83 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:23.995521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:24.149107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:24.152728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:24.153264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:24.153537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] PATH_INFO: `/v3/users/47ac37a64dce4131927de3a3f8205fe6` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:24.157161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ac37a64dce4131927de3a3f8205fe6)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:24.164772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:24.322834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ac37a64dce4131927de3a3f8205fe6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:24.347317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ac37a64dce4131927de3a3f8205fe6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:24.358887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ac37a64dce4131927de3a3f8205fe6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:24.360865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ac37a64dce4131927de3a3f8205fe6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:24.361641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ac37a64dce4131927de3a3f8205fe6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:24.409308 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 915/1816] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:23 2022] DELETE /identity/v3/users/47ac37a64dce4131927de3a3f8205fe6 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:25.522747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de50ec-e8cf-4c57-a293-c92d86850cec None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:25.523253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de50ec-e8cf-4c57-a293-c92d86850cec None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:25.523334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de50ec-e8cf-4c57-a293-c92d86850cec None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:25.524959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:25.525422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 902/1817] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:05:25.526523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:25.581945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb08ec6-adea-4586-83aa-e3290cafa453 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:25.582573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb08ec6-adea-4586-83aa-e3290cafa453 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:25.583067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb08ec6-adea-4586-83aa-e3290cafa453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:25.624335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3fb08ec6-adea-4586-83aa-e3290cafa453 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:25.626047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3fb08ec6-adea-4586-83aa-e3290cafa453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:25.703724 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbf728-8f97-4efb-9f56-183caa70f869 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:25.723825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:25.724657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:25.724946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:25.725221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] PATH_INFO: `/v3/projects/d3846e9ebd4c42fa8388f90d8d8ce16e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:25.726865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3846e9ebd4c42fa8388f90d8d8ce16e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:25.730706 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:25.884238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:25.889834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 903/1818] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:05:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:25.895543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3846e9ebd4c42fa8388f90d8d8ce16e', 'request_id': 'req-c5006931-8603-4ba9-8bf6-64ebd556076a'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:25.922009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3846e9ebd4c42fa8388f90d8d8ce16e', 'request_id': 'req-c5006931-8603-4ba9-8bf6-64ebd556076a'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:25.937378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3846e9ebd4c42fa8388f90d8d8ce16e', 'request_id': 'req-c5006931-8603-4ba9-8bf6-64ebd556076a'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:25.966021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] The token cache is being invalidate because project d3846e9ebd4c42fa8388f90d8d8ce16e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:25.966353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:25.969435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:25.977434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 916/1819] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:25 2022] DELETE /identity/v3/projects/d3846e9ebd4c42fa8388f90d8d8ce16e => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:25.978106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb08ec6-adea-4586-83aa-e3290cafa453 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:26.000845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb08ec6-adea-4586-83aa-e3290cafa453 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:26.104455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:26.170260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:26.171115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:26.171542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:26.171936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] PATH_INFO: `/v3/users/9545e0af8f644b9992f0db1b14ac2d3d` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:26.176912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9545e0af8f644b9992f0db1b14ac2d3d)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:26.179863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:26.296749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9545e0af8f644b9992f0db1b14ac2d3d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:26.299085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9545e0af8f644b9992f0db1b14ac2d3d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:26.320505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9545e0af8f644b9992f0db1b14ac2d3d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:26.350884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9545e0af8f644b9992f0db1b14ac2d3d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:26.366063 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9545e0af8f644b9992f0db1b14ac2d3d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:26.375874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 904/1820] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:25 2022] DELETE /identity/v3/users/9545e0af8f644b9992f0db1b14ac2d3d => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:27.421161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:27.425554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:27.546123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:27.549970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:27.756656 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0cad91-2f16-427e-b7ec-9cbf84822449 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:27.791772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:27.804181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:27.804540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:27.804820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] PATH_INFO: `/v3/projects/ba9d2afc92c147b3970004956c9ffae2` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:27.810630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9d2afc92c147b3970004956c9ffae2)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:27.822912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:27.826840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5006931-8603-4ba9-8bf6-64ebd556076a admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:27.939392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9d2afc92c147b3970004956c9ffae2', 'request_id': 'req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:27.948279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9d2afc92c147b3970004956c9ffae2', 'request_id': 'req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:27.951618 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:27.962331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9d2afc92c147b3970004956c9ffae2', 'request_id': 'req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:28.053473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] The token cache is being invalidate because project ba9d2afc92c147b3970004956c9ffae2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:28.053998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:28.058410 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-41c25b2a-ebfa-44b7-bd05-4fdf5a4ec49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:28.068759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 905/1821] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:27 2022] DELETE /identity/v3/projects/ba9d2afc92c147b3970004956c9ffae2 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:28.075172 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd4c56-c1b7-4285-9a05-796b0f07166d None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:28.076086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd4c56-c1b7-4285-9a05-796b0f07166d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:28.076587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd4c56-c1b7-4285-9a05-796b0f07166d None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:28.079681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 906/1822] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:05:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:05:28.081700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38a96-5439-4027-9d69-97dfe4bee226 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:28.082479 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38a96-5439-4027-9d69-97dfe4bee226 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:28.083786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38a96-5439-4027-9d69-97dfe4bee226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:28.155216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7fb38a96-5439-4027-9d69-97dfe4bee226 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:28.156495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7fb38a96-5439-4027-9d69-97dfe4bee226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:28.165723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55a26e-a431-4912-a95d-07f46535c953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:28.166474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55a26e-a431-4912-a95d-07f46535c953 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:28.166753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55a26e-a431-4912-a95d-07f46535c953 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:28.167028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55a26e-a431-4912-a95d-07f46535c953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:28.172642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5b55a26e-a431-4912-a95d-07f46535c953 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:28.204695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55a26e-a431-4912-a95d-07f46535c953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:28.206370 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55a26e-a431-4912-a95d-07f46535c953 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:28.246122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 917/1823] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 827 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:28.251444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55a26e-a431-4912-a95d-07f46535c953 service nova] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:28.258235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5b55a26e-a431-4912-a95d-07f46535c953 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:28.327011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55a26e-a431-4912-a95d-07f46535c953 service nova] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:28.412850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 907/1824] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:28.422500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50508de2-259d-4415-93ec-07302cc90652 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:28.423146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50508de2-259d-4415-93ec-07302cc90652 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:28.423480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50508de2-259d-4415-93ec-07302cc90652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:28.543712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-50508de2-259d-4415-93ec-07302cc90652 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:28.545397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-50508de2-259d-4415-93ec-07302cc90652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:28.595302 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:28.635162 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:28.635874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:28.636165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:28.636475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] PATH_INFO: `/v3/projects/65871cda029148faa8754a06b6b91bbb` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:28.638721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65871cda029148faa8754a06b6b91bbb)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:28.640816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:28.677938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 908/1825] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:05:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:28.680373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba56f9d-024e-4cb3-b337-6db0da9055e4 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:28.680921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba56f9d-024e-4cb3-b337-6db0da9055e4 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:28.682001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba56f9d-024e-4cb3-b337-6db0da9055e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:28.717979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9ba56f9d-024e-4cb3-b337-6db0da9055e4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:28.719648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9ba56f9d-024e-4cb3-b337-6db0da9055e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:28.740513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 909/1826] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:28.748186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba56f9d-024e-4cb3-b337-6db0da9055e4 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:28.761779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba56f9d-024e-4cb3-b337-6db0da9055e4 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:28.769332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b621758-ca8c-4b70-9c89-0f1f32d23e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:28.770137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621758-ca8c-4b70-9c89-0f1f32d23e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:28.771215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621758-ca8c-4b70-9c89-0f1f32d23e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:28.771492 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b621758-ca8c-4b70-9c89-0f1f32d23e64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:28.772486 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b621758-ca8c-4b70-9c89-0f1f32d23e64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:28.772956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b621758-ca8c-4b70-9c89-0f1f32d23e64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:28.776338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b621758-ca8c-4b70-9c89-0f1f32d23e64 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:28.795145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 910/1827] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:05:28.799959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b621758-ca8c-4b70-9c89-0f1f32d23e64 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:28.817137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b621758-ca8c-4b70-9c89-0f1f32d23e64 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:28.861063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65871cda029148faa8754a06b6b91bbb', 'request_id': 'req-1388d78c-605d-4aa7-aafc-43717bfe38b1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:28.871770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65871cda029148faa8754a06b6b91bbb', 'request_id': 'req-1388d78c-605d-4aa7-aafc-43717bfe38b1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:28.894396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65871cda029148faa8754a06b6b91bbb', 'request_id': 'req-1388d78c-605d-4aa7-aafc-43717bfe38b1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:28.916169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] The token cache is being invalidate because project 65871cda029148faa8754a06b6b91bbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:28.916876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:28.917699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:28.923223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 918/1828] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:28 2022] DELETE /identity/v3/projects/65871cda029148faa8754a06b6b91bbb => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:28.923223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:28.923223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:28.925242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:28.934980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301983ee-538a-4261-be79-1b433ac329d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:28.935854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301983ee-538a-4261-be79-1b433ac329d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:28.936566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301983ee-538a-4261-be79-1b433ac329d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:28.936855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301983ee-538a-4261-be79-1b433ac329d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:28.938599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301983ee-538a-4261-be79-1b433ac329d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:28.941090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301983ee-538a-4261-be79-1b433ac329d7 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:28.980346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 911/1829] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:28 2022] POST /identity/v3/projects => generated 377 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:28.982272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c6edd-9bc3-4526-a0d4-827e0fd5785c None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:28.982662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c6edd-9bc3-4526-a0d4-827e0fd5785c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:28.983182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c6edd-9bc3-4526-a0d4-827e0fd5785c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.004841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4c3c6edd-9bc3-4526-a0d4-827e0fd5785c None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:29.006285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4c3c6edd-9bc3-4526-a0d4-827e0fd5785c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:29.006822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388d78c-605d-4aa7-aafc-43717bfe38b1 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.077881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 912/1830] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:29.084626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c6edd-9bc3-4526-a0d4-827e0fd5785c None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.084626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4c3c6edd-9bc3-4526-a0d4-827e0fd5785c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:29.093324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.094763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c6edd-9bc3-4526-a0d4-827e0fd5785c None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.094890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.095062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.095450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] PATH_INFO: `/v3/users/49130731dcdb45bdac56b801318c6ea9` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.096793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49130731dcdb45bdac56b801318c6ea9)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.099029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.099759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f6075-bb90-4ff1-8755-444081661590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.100455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f6075-bb90-4ff1-8755-444081661590 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.100870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f6075-bb90-4ff1-8755-444081661590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.102177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f6075-bb90-4ff1-8755-444081661590 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.102177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f6075-bb90-4ff1-8755-444081661590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.103979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f6075-bb90-4ff1-8755-444081661590 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.132051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 913/1831] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:29 2022] POST /identity/v3/users => generated 289 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:29.135180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76cc6a-b40a-4bcd-a566-227d7303b1b3 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.135180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76cc6a-b40a-4bcd-a566-227d7303b1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.135476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76cc6a-b40a-4bcd-a566-227d7303b1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.150917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:29.163892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49130731dcdb45bdac56b801318c6ea9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:29.164816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-ad76cc6a-b40a-4bcd-a566-227d7303b1b3 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:29.166869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ad76cc6a-b40a-4bcd-a566-227d7303b1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:29.176347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 914/1832] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:29.179601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49130731dcdb45bdac56b801318c6ea9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:29.181115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76cc6a-b40a-4bcd-a566-227d7303b1b3 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.188063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49130731dcdb45bdac56b801318c6ea9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:29.188917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49130731dcdb45bdac56b801318c6ea9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:29.189547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49130731dcdb45bdac56b801318c6ea9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:29.192104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76cc6a-b40a-4bcd-a566-227d7303b1b3 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.198290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a4c5e-0cde-44a1-919e-889a9cbcfd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.199006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a4c5e-0cde-44a1-919e-889a9cbcfd81 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.199329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a4c5e-0cde-44a1-919e-889a9cbcfd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.199615 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a4c5e-0cde-44a1-919e-889a9cbcfd81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.201203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a4c5e-0cde-44a1-919e-889a9cbcfd81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.203040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a4c5e-0cde-44a1-919e-889a9cbcfd81 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.205857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 919/1833] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:28 2022] DELETE /identity/v3/users/49130731dcdb45bdac56b801318c6ea9 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:29.206931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.208190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:29.223984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 915/1834] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:29.226111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6514a6b-4add-42bb-a0dc-dcba33f65611 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.226111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6514a6b-4add-42bb-a0dc-dcba33f65611 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.226406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6514a6b-4add-42bb-a0dc-dcba33f65611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.251542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-f6514a6b-4add-42bb-a0dc-dcba33f65611 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:29.253635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f6514a6b-4add-42bb-a0dc-dcba33f65611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:29.312515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542a3ec-c39b-4963-9f45-fed1b34884d8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.326264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758eadf-415b-4838-8d5f-a6fe773199cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.329657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758eadf-415b-4838-8d5f-a6fe773199cd admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.330170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758eadf-415b-4838-8d5f-a6fe773199cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.331955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758eadf-415b-4838-8d5f-a6fe773199cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.334472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758eadf-415b-4838-8d5f-a6fe773199cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:29.335422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758eadf-415b-4838-8d5f-a6fe773199cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.337903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758eadf-415b-4838-8d5f-a6fe773199cd admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.354638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 920/1835] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:05:29.354638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758eadf-415b-4838-8d5f-a6fe773199cd admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.357690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a758eadf-415b-4838-8d5f-a6fe773199cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:29.371288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 916/1836] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:29.372609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6514a6b-4add-42bb-a0dc-dcba33f65611 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.388420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758eadf-415b-4838-8d5f-a6fe773199cd admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.394768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6514a6b-4add-42bb-a0dc-dcba33f65611 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.401752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b66a9-d7bc-4c94-9fda-1695326d4fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.401752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b66a9-d7bc-4c94-9fda-1695326d4fed admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.402051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887829b-0991-434c-88c1-7c4f96189d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.402160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b66a9-d7bc-4c94-9fda-1695326d4fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.402160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b66a9-d7bc-4c94-9fda-1695326d4fed admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.402293 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887829b-0991-434c-88c1-7c4f96189d31 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.402293 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887829b-0991-434c-88c1-7c4f96189d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.402293 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887829b-0991-434c-88c1-7c4f96189d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.402498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b66a9-d7bc-4c94-9fda-1695326d4fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.403127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887829b-0991-434c-88c1-7c4f96189d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.403732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b66a9-d7bc-4c94-9fda-1695326d4fed admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.405332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887829b-0991-434c-88c1-7c4f96189d31 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.420234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 921/1837] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:29.420234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b66a9-d7bc-4c94-9fda-1695326d4fed admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.421862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ef5b66a9-d7bc-4c94-9fda-1695326d4fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:29.440199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 917/1838] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:29 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:29.440199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887829b-0991-434c-88c1-7c4f96189d31 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.476887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:29.503994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b66a9-d7bc-4c94-9fda-1695326d4fed admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.539409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887829b-0991-434c-88c1-7c4f96189d31 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.545403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443ecad-5334-4eb8-bf9f-d07ae50761d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.546181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443ecad-5334-4eb8-bf9f-d07ae50761d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.546806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443ecad-5334-4eb8-bf9f-d07ae50761d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.546806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443ecad-5334-4eb8-bf9f-d07ae50761d2 admin admin] PATH_INFO: `/v3/projects/eacbbfc96d8c4633800d38d3ef0426f6/users/be895f2f4b274267a8560b647137c144/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.563474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443ecad-5334-4eb8-bf9f-d07ae50761d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eacbbfc96d8c4633800d38d3ef0426f6, user_id=be895f2f4b274267a8560b647137c144, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.566223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443ecad-5334-4eb8-bf9f-d07ae50761d2 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.574829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce8e7e-e0d8-4852-8441-9cc372d693c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.575596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce8e7e-e0d8-4852-8441-9cc372d693c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.578040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce8e7e-e0d8-4852-8441-9cc372d693c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.578327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce8e7e-e0d8-4852-8441-9cc372d693c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.591491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 918/1839] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:29 2022] PUT /identity/v3/projects/eacbbfc96d8c4633800d38d3ef0426f6/users/be895f2f4b274267a8560b647137c144/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:29.592997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443ecad-5334-4eb8-bf9f-d07ae50761d2 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.606293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:29.610660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce8e7e-e0d8-4852-8441-9cc372d693c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.613096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce8e7e-e0d8-4852-8441-9cc372d693c1 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.626977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 922/1840] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:29.628199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce8e7e-e0d8-4852-8441-9cc372d693c1 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.629799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9cce8e7e-e0d8-4852-8441-9cc372d693c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:29.678348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443ecad-5334-4eb8-bf9f-d07ae50761d2 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.684751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e650394-a4ba-4166-a992-da69ac6863bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.685547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e650394-a4ba-4166-a992-da69ac6863bd admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.685925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e650394-a4ba-4166-a992-da69ac6863bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.686337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e650394-a4ba-4166-a992-da69ac6863bd admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.687090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e650394-a4ba-4166-a992-da69ac6863bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.689351 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e650394-a4ba-4166-a992-da69ac6863bd admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.719300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 919/1841] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:29 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:29.722581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb118bc-e861-4b25-944b-49db9a9c1394 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.722764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb118bc-e861-4b25-944b-49db9a9c1394 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.723021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb118bc-e861-4b25-944b-49db9a9c1394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.763773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce8e7e-e0d8-4852-8441-9cc372d693c1 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.766140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-afb118bc-e861-4b25-944b-49db9a9c1394 None None] MFA Rules not processed for user `be895f2f4b274267a8560b647137c144`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:29.767715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-afb118bc-e861-4b25-944b-49db9a9c1394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:29.851110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 920/1842] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:29.851668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdce966-75a6-48fb-85cb-911f4391fe57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.852524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdce966-75a6-48fb-85cb-911f4391fe57 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.852907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdce966-75a6-48fb-85cb-911f4391fe57 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.853282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdce966-75a6-48fb-85cb-911f4391fe57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.855535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb118bc-e861-4b25-944b-49db9a9c1394 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.856059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-bbdce966-75a6-48fb-85cb-911f4391fe57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:29.869217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb118bc-e861-4b25-944b-49db9a9c1394 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.870298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdce966-75a6-48fb-85cb-911f4391fe57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.872336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdce966-75a6-48fb-85cb-911f4391fe57 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.875294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9686360-91b6-4576-911f-590beef2d51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.876060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9686360-91b6-4576-911f-590beef2d51d admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.876313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9686360-91b6-4576-911f-590beef2d51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.876766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9686360-91b6-4576-911f-590beef2d51d admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.877758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9686360-91b6-4576-911f-590beef2d51d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.879704 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9686360-91b6-4576-911f-590beef2d51d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.887743 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 923/1843] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:29.889022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdce966-75a6-48fb-85cb-911f4391fe57 service nova] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.901036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdce966-75a6-48fb-85cb-911f4391fe57 service nova] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.905155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 921/1844] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:29.909183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9686360-91b6-4576-911f-590beef2d51d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.910242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0deaa2e-424e-48fa-a4b7-7de438f5baa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.911036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0deaa2e-424e-48fa-a4b7-7de438f5baa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.911491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0deaa2e-424e-48fa-a4b7-7de438f5baa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.912240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0deaa2e-424e-48fa-a4b7-7de438f5baa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.913325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0deaa2e-424e-48fa-a4b7-7de438f5baa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.915484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0deaa2e-424e-48fa-a4b7-7de438f5baa9 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.929124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9686360-91b6-4576-911f-590beef2d51d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:29.936005 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61eb2a5-86fa-401d-b968-6dafb7669fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:29.936594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61eb2a5-86fa-401d-b968-6dafb7669fad admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:29.936999 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61eb2a5-86fa-401d-b968-6dafb7669fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:29.937328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61eb2a5-86fa-401d-b968-6dafb7669fad admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:29.949251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61eb2a5-86fa-401d-b968-6dafb7669fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:29.951339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61eb2a5-86fa-401d-b968-6dafb7669fad admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:29.955276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 924/1845] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:29 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:29.956014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0deaa2e-424e-48fa-a4b7-7de438f5baa9 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.970798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 922/1846] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:29.976164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61eb2a5-86fa-401d-b968-6dafb7669fad admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:29.983632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:29.989662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61eb2a5-86fa-401d-b968-6dafb7669fad admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.037845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:30.046466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0deaa2e-424e-48fa-a4b7-7de438f5baa9 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.053545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300690b-7df1-4426-9f7e-82790983d0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.054224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300690b-7df1-4426-9f7e-82790983d0dd admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.054511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300690b-7df1-4426-9f7e-82790983d0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.054789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300690b-7df1-4426-9f7e-82790983d0dd admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.055466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300690b-7df1-4426-9f7e-82790983d0dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.057519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300690b-7df1-4426-9f7e-82790983d0dd admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.073318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964ee70-2f59-47e9-bfa0-1ef2de781546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.074284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964ee70-2f59-47e9-bfa0-1ef2de781546 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.074570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964ee70-2f59-47e9-bfa0-1ef2de781546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.074854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964ee70-2f59-47e9-bfa0-1ef2de781546 admin admin] PATH_INFO: `/v3/projects/e2f831065d8c4b27b77e8bfbaf50f7f3/users/4784dd7e7e364c7c808b9ce17c9ab409/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.083503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 925/1847] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:29 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:30.087279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300690b-7df1-4426-9f7e-82790983d0dd admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.088990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7300690b-7df1-4426-9f7e-82790983d0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:30.090889 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964ee70-2f59-47e9-bfa0-1ef2de781546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f831065d8c4b27b77e8bfbaf50f7f3, user_id=4784dd7e7e364c7c808b9ce17c9ab409, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.093767 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964ee70-2f59-47e9-bfa0-1ef2de781546 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.102057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300690b-7df1-4426-9f7e-82790983d0dd admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.109149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 923/1848] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:29 2022] PUT /identity/v3/projects/e2f831065d8c4b27b77e8bfbaf50f7f3/users/4784dd7e7e364c7c808b9ce17c9ab409/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:30.110002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964ee70-2f59-47e9-bfa0-1ef2de781546 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.162477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.163251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.163532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.163804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] PATH_INFO: `/v3/projects/9696f4282dad423cb8e0bb53ac931bb9` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.164975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9696f4282dad423cb8e0bb53ac931bb9)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.166892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.177933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964ee70-2f59-47e9-bfa0-1ef2de781546 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.183491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a9c40-cccb-4387-83dd-096fb8a1b7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.184071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a9c40-cccb-4387-83dd-096fb8a1b7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.184372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a9c40-cccb-4387-83dd-096fb8a1b7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.184644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a9c40-cccb-4387-83dd-096fb8a1b7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.185338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a9c40-cccb-4387-83dd-096fb8a1b7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.186967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a9c40-cccb-4387-83dd-096fb8a1b7c7 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.202947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 924/1849] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:30.202947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a70b-65a6-4eb6-81dc-905583d95ed1 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.202947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a70b-65a6-4eb6-81dc-905583d95ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.202947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1a70b-65a6-4eb6-81dc-905583d95ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.231758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9696f4282dad423cb8e0bb53ac931bb9', 'request_id': 'req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:30.238216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-dfb1a70b-65a6-4eb6-81dc-905583d95ed1 None None] MFA Rules not processed for user `4784dd7e7e364c7c808b9ce17c9ab409`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:30.238437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9696f4282dad423cb8e0bb53ac931bb9', 'request_id': 'req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:30.239597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-dfb1a70b-65a6-4eb6-81dc-905583d95ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:30.247890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9696f4282dad423cb8e0bb53ac931bb9', 'request_id': 'req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:30.259071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] The token cache is being invalidate because project 9696f4282dad423cb8e0bb53ac931bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:30.259353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:30.259992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:30.261580 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 926/1850] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:30 2022] DELETE /identity/v3/projects/9696f4282dad423cb8e0bb53ac931bb9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:30.265090 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.266508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:30.308125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 925/1851] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:30.309122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1a70b-65a6-4eb6-81dc-905583d95ed1 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.310412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-dfb1a70b-65a6-4eb6-81dc-905583d95ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:30.327302 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:30.341429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fd226-c879-4bc3-9bcc-e673ae1e4eb5 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.347092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12366682-e218-4605-ba2a-a65dabf2139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.347974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12366682-e218-4605-ba2a-a65dabf2139d admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.348175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12366682-e218-4605-ba2a-a65dabf2139d admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.348434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12366682-e218-4605-ba2a-a65dabf2139d admin admin] PATH_INFO: `/v3/projects/b498ea92e45047399fe857777a43bb8c/users/5b76d197cceb4c25bb17c74713ba8eb7/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.370400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12366682-e218-4605-ba2a-a65dabf2139d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b498ea92e45047399fe857777a43bb8c, user_id=5b76d197cceb4c25bb17c74713ba8eb7, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.373275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12366682-e218-4605-ba2a-a65dabf2139d admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.383864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:30.392886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 927/1852] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:30 2022] PUT /identity/v3/projects/b498ea92e45047399fe857777a43bb8c/users/5b76d197cceb4c25bb17c74713ba8eb7/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:30.393923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12366682-e218-4605-ba2a-a65dabf2139d admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.395707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-12366682-e218-4605-ba2a-a65dabf2139d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:30.409383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1a70b-65a6-4eb6-81dc-905583d95ed1 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.474354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12366682-e218-4605-ba2a-a65dabf2139d admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.479604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.479819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad957b1-07dc-4034-bb9b-27f9cf9d7c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.480525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad957b1-07dc-4034-bb9b-27f9cf9d7c40 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.480632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.480758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad957b1-07dc-4034-bb9b-27f9cf9d7c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.480991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.481101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad957b1-07dc-4034-bb9b-27f9cf9d7c40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.481394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] PATH_INFO: `/v3/users/ddf66429b45a40cc82a64c35163ed854` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.481707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad957b1-07dc-4034-bb9b-27f9cf9d7c40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.482739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddf66429b45a40cc82a64c35163ed854)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.483758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad957b1-07dc-4034-bb9b-27f9cf9d7c40 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.485082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.517341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 928/1853] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:30 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:30.519074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad957b1-07dc-4034-bb9b-27f9cf9d7c40 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.570370 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddf66429b45a40cc82a64c35163ed854'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:30.571955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddf66429b45a40cc82a64c35163ed854'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:30.589408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddf66429b45a40cc82a64c35163ed854'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:30.613710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddf66429b45a40cc82a64c35163ed854'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:30.623109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddf66429b45a40cc82a64c35163ed854'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:30.630505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad957b1-07dc-4034-bb9b-27f9cf9d7c40 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.633951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 926/1854] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:30 2022] DELETE /identity/v3/users/ddf66429b45a40cc82a64c35163ed854 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:30.635154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.637194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:30.694062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:30.715420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8837086-4e7a-4226-8049-df27c91fe41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.716736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8837086-4e7a-4226-8049-df27c91fe41e admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.717094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8837086-4e7a-4226-8049-df27c91fe41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.717441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8837086-4e7a-4226-8049-df27c91fe41e admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.718226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8837086-4e7a-4226-8049-df27c91fe41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.720560 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8837086-4e7a-4226-8049-df27c91fe41e admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.726759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e71225-026b-4e7a-ad09-4276fdcf6671 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.733306 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556df5f-3384-4d88-af39-ced0f57bb99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.734332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556df5f-3384-4d88-af39-ced0f57bb99f admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.734842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556df5f-3384-4d88-af39-ced0f57bb99f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.735417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556df5f-3384-4d88-af39-ced0f57bb99f admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.736402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556df5f-3384-4d88-af39-ced0f57bb99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.736515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 929/1855] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:30.739100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556df5f-3384-4d88-af39-ced0f57bb99f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.741480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8837086-4e7a-4226-8049-df27c91fe41e admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.744422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e8837086-4e7a-4226-8049-df27c91fe41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:30.766097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 927/1856] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:30 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:30.770194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556df5f-3384-4d88-af39-ced0f57bb99f admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.770325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8837086-4e7a-4226-8049-df27c91fe41e admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.775472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5eac59-41ee-438f-82d9-9a85089ca594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.776418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eac59-41ee-438f-82d9-9a85089ca594 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.776704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eac59-41ee-438f-82d9-9a85089ca594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.776984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eac59-41ee-438f-82d9-9a85089ca594 admin admin] PATH_INFO: `/v3/projects/b498ea92e45047399fe857777a43bb8c/users/5b76d197cceb4c25bb17c74713ba8eb7/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.779007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eac59-41ee-438f-82d9-9a85089ca594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b498ea92e45047399fe857777a43bb8c, user_id=5b76d197cceb4c25bb17c74713ba8eb7, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.782760 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eac59-41ee-438f-82d9-9a85089ca594 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.789864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556df5f-3384-4d88-af39-ced0f57bb99f admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.798703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 930/1857] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:30 2022] PUT /identity/v3/projects/b498ea92e45047399fe857777a43bb8c/users/5b76d197cceb4c25bb17c74713ba8eb7/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:30.812580 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701373f6-45b8-4f0f-af42-730e82d57758 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.812924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701373f6-45b8-4f0f-af42-730e82d57758 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.813138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701373f6-45b8-4f0f-af42-730e82d57758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.873069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:30.883985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-701373f6-45b8-4f0f-af42-730e82d57758 None None] MFA Rules not processed for user `5b76d197cceb4c25bb17c74713ba8eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:30.885473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-701373f6-45b8-4f0f-af42-730e82d57758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:30.894580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0fae2-a742-49ed-a18d-df60f6fdab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.895619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0fae2-a742-49ed-a18d-df60f6fdab10 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.896169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0fae2-a742-49ed-a18d-df60f6fdab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.905320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0fae2-a742-49ed-a18d-df60f6fdab10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.906243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0fae2-a742-49ed-a18d-df60f6fdab10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.909739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0fae2-a742-49ed-a18d-df60f6fdab10 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.939389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 928/1858] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:30.941097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0fae2-a742-49ed-a18d-df60f6fdab10 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.952254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0fae2-a742-49ed-a18d-df60f6fdab10 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.957608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85def5bc-9a41-4e71-b5f4-c549c47a04d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.958040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85def5bc-9a41-4e71-b5f4-c549c47a04d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.958317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85def5bc-9a41-4e71-b5f4-c549c47a04d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:30.958604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85def5bc-9a41-4e71-b5f4-c549c47a04d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:30.959304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85def5bc-9a41-4e71-b5f4-c549c47a04d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:30.961070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85def5bc-9a41-4e71-b5f4-c549c47a04d2 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:30.971954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 931/1859] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:30.976080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 929/1860] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:30.981329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701373f6-45b8-4f0f-af42-730e82d57758 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.983942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85def5bc-9a41-4e71-b5f4-c549c47a04d2 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:30.985943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-85def5bc-9a41-4e71-b5f4-c549c47a04d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:30.993835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701373f6-45b8-4f0f-af42-730e82d57758 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:30.998890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3387dc-0713-4b82-a613-faf0951b022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:30.999591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3387dc-0713-4b82-a613-faf0951b022b admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:30.999896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3387dc-0713-4b82-a613-faf0951b022b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.000137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3387dc-0713-4b82-a613-faf0951b022b admin admin] PATH_INFO: `/v3/projects/53ae198cf5ec477ea68baacfa29c29a5/users/ea7133129e7342b786dd3f5539bd0318/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.018895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3387dc-0713-4b82-a613-faf0951b022b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ae198cf5ec477ea68baacfa29c29a5, user_id=ea7133129e7342b786dd3f5539bd0318, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.021994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3387dc-0713-4b82-a613-faf0951b022b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.040695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 932/1861] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:30 2022] PUT /identity/v3/projects/53ae198cf5ec477ea68baacfa29c29a5/users/ea7133129e7342b786dd3f5539bd0318/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:31.041672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3387dc-0713-4b82-a613-faf0951b022b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.043249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-4e3387dc-0713-4b82-a613-faf0951b022b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:31.062574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85def5bc-9a41-4e71-b5f4-c549c47a04d2 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.112386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c0000-7395-492f-aa50-40fa0e2f7cb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.112655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3387dc-0713-4b82-a613-faf0951b022b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.113048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c0000-7395-492f-aa50-40fa0e2f7cb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.113401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c0000-7395-492f-aa50-40fa0e2f7cb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.113677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c0000-7395-492f-aa50-40fa0e2f7cb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.115559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-de6c0000-7395-492f-aa50-40fa0e2f7cb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:31.117657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.118698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.118980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.119259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] PATH_INFO: `/v3/projects/beb17525c94443b8b747fbcd7aa53f33` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.120427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=beb17525c94443b8b747fbcd7aa53f33)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.122452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.142433 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:31.172205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beb17525c94443b8b747fbcd7aa53f33', 'request_id': 'req-508d2a5c-6530-4cd8-b553-e77dae64c2ac'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:31.177298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c0000-7395-492f-aa50-40fa0e2f7cb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.178861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c0000-7395-492f-aa50-40fa0e2f7cb9 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.179260 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beb17525c94443b8b747fbcd7aa53f33', 'request_id': 'req-508d2a5c-6530-4cd8-b553-e77dae64c2ac'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:31.187455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beb17525c94443b8b747fbcd7aa53f33', 'request_id': 'req-508d2a5c-6530-4cd8-b553-e77dae64c2ac'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:31.188167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 930/1862] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:31.189269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049b5fb-e9ac-404b-9556-68db744878e2 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.189551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049b5fb-e9ac-404b-9556-68db744878e2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.189851 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8049b5fb-e9ac-404b-9556-68db744878e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.200491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] The token cache is being invalidate because project beb17525c94443b8b747fbcd7aa53f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:31.200791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:31.201528 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-508d2a5c-6530-4cd8-b553-e77dae64c2ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:31.203026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 933/1863] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:31 2022] DELETE /identity/v3/projects/beb17525c94443b8b747fbcd7aa53f33 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:31.209427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d528d2-293e-480f-b360-b405c09911f1 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.209866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d528d2-293e-480f-b360-b405c09911f1 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.210169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d528d2-293e-480f-b360-b405c09911f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.220654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8049b5fb-e9ac-404b-9556-68db744878e2 None None] MFA Rules not processed for user `ea7133129e7342b786dd3f5539bd0318`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:31.221116 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:31.221948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8049b5fb-e9ac-404b-9556-68db744878e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:31.234381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e6d528d2-293e-480f-b360-b405c09911f1 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:31.236595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e6d528d2-293e-480f-b360-b405c09911f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:31.300263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 931/1864] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:31.305179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049b5fb-e9ac-404b-9556-68db744878e2 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.306609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8049b5fb-e9ac-404b-9556-68db744878e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:31.308874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 934/1865] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:31.317038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce6b49-2c87-4a6b-aa65-cd6ca2fba80d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.317329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce6b49-2c87-4a6b-aa65-cd6ca2fba80d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.317610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce6b49-2c87-4a6b-aa65-cd6ca2fba80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.320687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8049b5fb-e9ac-404b-9556-68db744878e2 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.325793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4185b3-48f2-41b2-a9f6-f807629ae9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.327253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4185b3-48f2-41b2-a9f6-f807629ae9aa admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.327530 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4185b3-48f2-41b2-a9f6-f807629ae9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.327900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4185b3-48f2-41b2-a9f6-f807629ae9aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.329163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4185b3-48f2-41b2-a9f6-f807629ae9aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.332665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4185b3-48f2-41b2-a9f6-f807629ae9aa admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.337621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1fce6b49-2c87-4a6b-aa65-cd6ca2fba80d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:31.338944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1fce6b49-2c87-4a6b-aa65-cd6ca2fba80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:31.347348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 935/1866] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:31.352449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce6b49-2c87-4a6b-aa65-cd6ca2fba80d None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.363996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce6b49-2c87-4a6b-aa65-cd6ca2fba80d None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.367632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 932/1867] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:31 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:31.370794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8738a-5281-4669-b569-2d86b8f9a43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.371492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8738a-5281-4669-b569-2d86b8f9a43c admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.371771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8738a-5281-4669-b569-2d86b8f9a43c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.372039 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8738a-5281-4669-b569-2d86b8f9a43c admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.372617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4185b3-48f2-41b2-a9f6-f807629ae9aa admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.372771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8738a-5281-4669-b569-2d86b8f9a43c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:31.373118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8738a-5281-4669-b569-2d86b8f9a43c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.374777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8738a-5281-4669-b569-2d86b8f9a43c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.389519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 936/1868] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:05:31.394359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8738a-5281-4669-b569-2d86b8f9a43c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.410680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8738a-5281-4669-b569-2d86b8f9a43c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.509755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4185b3-48f2-41b2-a9f6-f807629ae9aa admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.517494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce0db6-281a-4d07-9a30-ad136d70e63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.517706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2028a-4d89-43df-b761-de505045cb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.518730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce0db6-281a-4d07-9a30-ad136d70e63b admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.518850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2028a-4d89-43df-b761-de505045cb37 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.519359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce0db6-281a-4d07-9a30-ad136d70e63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.520499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce0db6-281a-4d07-9a30-ad136d70e63b admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.520923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2028a-4d89-43df-b761-de505045cb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.521871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2028a-4d89-43df-b761-de505045cb37 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.522238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce0db6-281a-4d07-9a30-ad136d70e63b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.523314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2028a-4d89-43df-b761-de505045cb37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.526049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce0db6-281a-4d07-9a30-ad136d70e63b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.527462 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c2028a-4d89-43df-b761-de505045cb37 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.563562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 933/1869] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:31 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:31.567342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2028a-4d89-43df-b761-de505045cb37 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.582340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 937/1870] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:31 2022] POST /identity/v3/projects => generated 369 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:31.582490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2028a-4d89-43df-b761-de505045cb37 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.586187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce0db6-281a-4d07-9a30-ad136d70e63b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.615296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:31.668203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ce3e2-4baf-4881-b0ea-e8551a86e217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.669008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ce3e2-4baf-4881-b0ea-e8551a86e217 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.669416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ce3e2-4baf-4881-b0ea-e8551a86e217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.669759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ce3e2-4baf-4881-b0ea-e8551a86e217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.670607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ce3e2-4baf-4881-b0ea-e8551a86e217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.673164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ce3e2-4baf-4881-b0ea-e8551a86e217 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.677323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce0db6-281a-4d07-9a30-ad136d70e63b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.683283 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed105ab8-9ba1-4d56-9ba5-edbe31f22aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.684053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105ab8-9ba1-4d56-9ba5-edbe31f22aac admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.684437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105ab8-9ba1-4d56-9ba5-edbe31f22aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.684786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105ab8-9ba1-4d56-9ba5-edbe31f22aac admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.685536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed105ab8-9ba1-4d56-9ba5-edbe31f22aac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.686795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 934/1871] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:05:31.687721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed105ab8-9ba1-4d56-9ba5-edbe31f22aac admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.691003 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ce3e2-4baf-4881-b0ea-e8551a86e217 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.703266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ce3e2-4baf-4881-b0ea-e8551a86e217 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.708217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c2b69-5e7f-4746-8463-a976ed273050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.708980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c2b69-5e7f-4746-8463-a976ed273050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.709253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c2b69-5e7f-4746-8463-a976ed273050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.709577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c2b69-5e7f-4746-8463-a976ed273050 admin admin] PATH_INFO: `/v3/projects/8490b44172be4c98b057e3dab45a04a3/users/4a05704980a844909dfdb9bad0c44dd0/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.711097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:31.719017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 938/1872] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:31 2022] POST /identity/v3/users => generated 285 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:31.729936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c2b69-5e7f-4746-8463-a976ed273050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8490b44172be4c98b057e3dab45a04a3, user_id=4a05704980a844909dfdb9bad0c44dd0, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.734516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed105ab8-9ba1-4d56-9ba5-edbe31f22aac admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.735606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c2b69-5e7f-4746-8463-a976ed273050 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.754403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 935/1873] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:31 2022] PUT /identity/v3/projects/8490b44172be4c98b057e3dab45a04a3/users/4a05704980a844909dfdb9bad0c44dd0/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:31.755470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed105ab8-9ba1-4d56-9ba5-edbe31f22aac admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.760017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c2b69-5e7f-4746-8463-a976ed273050 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.841792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f61b45-6c8d-41c0-9940-2a38b579d0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.843182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f61b45-6c8d-41c0-9940-2a38b579d0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.843474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f61b45-6c8d-41c0-9940-2a38b579d0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.843807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f61b45-6c8d-41c0-9940-2a38b579d0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.845617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f61b45-6c8d-41c0-9940-2a38b579d0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.848018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f61b45-6c8d-41c0-9940-2a38b579d0b6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.861430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c2b69-5e7f-4746-8463-a976ed273050 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.868959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c80eae-78da-4286-b46b-8f3669b34873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.869874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c80eae-78da-4286-b46b-8f3669b34873 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.870255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c80eae-78da-4286-b46b-8f3669b34873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.870385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 939/1874] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:31.870705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c80eae-78da-4286-b46b-8f3669b34873 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.871271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f61b45-6c8d-41c0-9940-2a38b579d0b6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.871613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c80eae-78da-4286-b46b-8f3669b34873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.873791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c80eae-78da-4286-b46b-8f3669b34873 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.889072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f61b45-6c8d-41c0-9940-2a38b579d0b6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.898846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 936/1875] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:31.905656 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c80eae-78da-4286-b46b-8f3669b34873 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.905857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212eadb-6002-4ab1-b385-f976f8a64f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.906594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212eadb-6002-4ab1-b385-f976f8a64f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.907025 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212eadb-6002-4ab1-b385-f976f8a64f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.907329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212eadb-6002-4ab1-b385-f976f8a64f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.908266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212eadb-6002-4ab1-b385-f976f8a64f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.910621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212eadb-6002-4ab1-b385-f976f8a64f08 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.922283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c80eae-78da-4286-b46b-8f3669b34873 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:31.930473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc2de2e-44e9-4322-9166-82c87ed28382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:31.934903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2de2e-44e9-4322-9166-82c87ed28382 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.935398 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2de2e-44e9-4322-9166-82c87ed28382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.935744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc2de2e-44e9-4322-9166-82c87ed28382 admin admin] PATH_INFO: `/v3/projects/8490b44172be4c98b057e3dab45a04a3/users/4a05704980a844909dfdb9bad0c44dd0/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.938814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2de2e-44e9-4322-9166-82c87ed28382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8490b44172be4c98b057e3dab45a04a3, user_id=4a05704980a844909dfdb9bad0c44dd0, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:31.943307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc2de2e-44e9-4322-9166-82c87ed28382 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:31.948544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 940/1876] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:31.953453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212eadb-6002-4ab1-b385-f976f8a64f08 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:31.963899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 937/1877] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:31 2022] PUT /identity/v3/projects/8490b44172be4c98b057e3dab45a04a3/users/4a05704980a844909dfdb9bad0c44dd0/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:31.978344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2b4c7-2944-470e-bdc6-82cfeaae5ea6 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:31.978525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2b4c7-2944-470e-bdc6-82cfeaae5ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:31.978792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2b4c7-2944-470e-bdc6-82cfeaae5ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:31.983540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212eadb-6002-4ab1-b385-f976f8a64f08 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:32.049119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8de2b4c7-2944-470e-bdc6-82cfeaae5ea6 None None] MFA Rules not processed for user `4a05704980a844909dfdb9bad0c44dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:32.050415 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8de2b4c7-2944-470e-bdc6-82cfeaae5ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:32.140204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c1a73-cd70-4a84-b679-61182df0c47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:32.141851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1a73-cd70-4a84-b679-61182df0c47b admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:32.142528 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1a73-cd70-4a84-b679-61182df0c47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:32.142826 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c1a73-cd70-4a84-b679-61182df0c47b admin admin] PATH_INFO: `/v3/projects/d90139165e274a93a72f949e6909294b/users/39615e87fc7741298f99ab9045b1c969/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:32.163722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1a73-cd70-4a84-b679-61182df0c47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90139165e274a93a72f949e6909294b, user_id=39615e87fc7741298f99ab9045b1c969, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:32.167724 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c1a73-cd70-4a84-b679-61182df0c47b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:32.186080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 941/1878] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:31 2022] PUT /identity/v3/projects/d90139165e274a93a72f949e6909294b/users/39615e87fc7741298f99ab9045b1c969/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:32.190299 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 938/1879] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:32.197107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b2c7-fc0b-4a2a-9bf0-bee37c8863c2 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:32.197391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b2c7-fc0b-4a2a-9bf0-bee37c8863c2 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:32.197660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b2c7-fc0b-4a2a-9bf0-bee37c8863c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:32.202355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2b4c7-2944-470e-bdc6-82cfeaae5ea6 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:32.204470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8de2b4c7-2944-470e-bdc6-82cfeaae5ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:32.210640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:32.238833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-7d31b2c7-fc0b-4a2a-9bf0-bee37c8863c2 None None] MFA Rules not processed for user `39615e87fc7741298f99ab9045b1c969`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:32.240083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7d31b2c7-fc0b-4a2a-9bf0-bee37c8863c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:32.311291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2b4c7-2944-470e-bdc6-82cfeaae5ea6 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:32.375282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:32.385772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 942/1880] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:32.388012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31b2c7-fc0b-4a2a-9bf0-bee37c8863c2 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:32.413841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a195cf-58e8-41d8-88a3-64e9ff4f1120 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:32.414982 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a195cf-58e8-41d8-88a3-64e9ff4f1120 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:32.415514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a195cf-58e8-41d8-88a3-64e9ff4f1120 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:32.419471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a195cf-58e8-41d8-88a3-64e9ff4f1120 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:32.520036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31b2c7-fc0b-4a2a-9bf0-bee37c8863c2 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:32.525303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a195cf-58e8-41d8-88a3-64e9ff4f1120 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:32.525441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8aead8-7946-4873-8cda-cdd1a78c02e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:32.526170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8aead8-7946-4873-8cda-cdd1a78c02e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:32.526611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8aead8-7946-4873-8cda-cdd1a78c02e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:32.526780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a195cf-58e8-41d8-88a3-64e9ff4f1120 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:32.526884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8aead8-7946-4873-8cda-cdd1a78c02e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:32.527592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8aead8-7946-4873-8cda-cdd1a78c02e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:32.529917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8aead8-7946-4873-8cda-cdd1a78c02e0 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:32.537252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 939/1881] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:05:32.565856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 943/1882] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:32 2022] POST /identity/v3/projects => generated 369 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:32.570152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a195cf-58e8-41d8-88a3-64e9ff4f1120 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:32.589597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a195cf-58e8-41d8-88a3-64e9ff4f1120 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:32.595011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8aead8-7946-4873-8cda-cdd1a78c02e0 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:32.677073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8aead8-7946-4873-8cda-cdd1a78c02e0 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:32.681245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcaeac-a903-4750-a786-362efef0a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:32.681900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcaeac-a903-4750-a786-362efef0a84f admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:32.682357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcaeac-a903-4750-a786-362efef0a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:32.682636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcaeac-a903-4750-a786-362efef0a84f admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:32.683012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f6ec8-6bd7-4540-90fe-be21c9dfe5d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:32.683495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcaeac-a903-4750-a786-362efef0a84f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:32.684400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6ec8-6bd7-4540-90fe-be21c9dfe5d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:32.684690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6ec8-6bd7-4540-90fe-be21c9dfe5d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:32.684965 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f6ec8-6bd7-4540-90fe-be21c9dfe5d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:32.686178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbcaeac-a903-4750-a786-362efef0a84f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:32.686894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c02f6ec8-6bd7-4540-90fe-be21c9dfe5d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:32.707546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:32.719223 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 940/1883] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:32 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:32.720358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcaeac-a903-4750-a786-362efef0a84f admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:32.734542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcaeac-a903-4750-a786-362efef0a84f admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:32.739891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04ed02-eb58-4cc2-aa5a-82034a77b5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:32.740567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04ed02-eb58-4cc2-aa5a-82034a77b5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:32.740852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04ed02-eb58-4cc2-aa5a-82034a77b5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:32.741134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04ed02-eb58-4cc2-aa5a-82034a77b5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:32.741993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04ed02-eb58-4cc2-aa5a-82034a77b5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:32.744010 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04ed02-eb58-4cc2-aa5a-82034a77b5b9 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:32.759242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 941/1884] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:32.766150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04ed02-eb58-4cc2-aa5a-82034a77b5b9 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:32.777372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f6ec8-6bd7-4540-90fe-be21c9dfe5d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:32.778878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f6ec8-6bd7-4540-90fe-be21c9dfe5d5 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:32.779503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04ed02-eb58-4cc2-aa5a-82034a77b5b9 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:32.787584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e6db1-c015-4a3b-9b79-447fbf722f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:32.790547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6db1-c015-4a3b-9b79-447fbf722f2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:32.790547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6db1-c015-4a3b-9b79-447fbf722f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:32.790547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e6db1-c015-4a3b-9b79-447fbf722f2c admin admin] PATH_INFO: `/v3/projects/1b53ce99e71a4e90acd83cfdfd2f4c14/users/a05030128ac446fcb09bde4268e56603/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:32.793742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 944/1885] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:05:32.820752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e6db1-c015-4a3b-9b79-447fbf722f2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b53ce99e71a4e90acd83cfdfd2f4c14, user_id=a05030128ac446fcb09bde4268e56603, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:32.824511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e6db1-c015-4a3b-9b79-447fbf722f2c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:32.844818 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 942/1886] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:32 2022] PUT /identity/v3/projects/1b53ce99e71a4e90acd83cfdfd2f4c14/users/a05030128ac446fcb09bde4268e56603/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:32.849681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f6ec8-6bd7-4540-90fe-be21c9dfe5d5 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:32.868151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f6ec8-6bd7-4540-90fe-be21c9dfe5d5 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:32.940430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:32.979576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398bd36-3084-4aa8-ac1b-025d1afbf7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:32.980260 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398bd36-3084-4aa8-ac1b-025d1afbf7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:32.980575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398bd36-3084-4aa8-ac1b-025d1afbf7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:32.980850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398bd36-3084-4aa8-ac1b-025d1afbf7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:32.981591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398bd36-3084-4aa8-ac1b-025d1afbf7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:32.983269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398bd36-3084-4aa8-ac1b-025d1afbf7f0 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:32.998071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 945/1887] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:32.999474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e6db1-c015-4a3b-9b79-447fbf722f2c admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:33.013356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e6db1-c015-4a3b-9b79-447fbf722f2c admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:33.018571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9f38d-d190-41ad-ba4d-55857ecbb750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:33.019265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f38d-d190-41ad-ba4d-55857ecbb750 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:33.019568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f38d-d190-41ad-ba4d-55857ecbb750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:33.019940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9f38d-d190-41ad-ba4d-55857ecbb750 admin admin] PATH_INFO: `/v3/projects/1b53ce99e71a4e90acd83cfdfd2f4c14/users/a05030128ac446fcb09bde4268e56603/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:33.022106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9f38d-d190-41ad-ba4d-55857ecbb750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b53ce99e71a4e90acd83cfdfd2f4c14, user_id=a05030128ac446fcb09bde4268e56603, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:33.024877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9f38d-d190-41ad-ba4d-55857ecbb750 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:33.037851 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 943/1888] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:32 2022] PUT /identity/v3/projects/1b53ce99e71a4e90acd83cfdfd2f4c14/users/a05030128ac446fcb09bde4268e56603/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:33.050084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afd545-3a56-4890-b2e9-e3dfc5ff4631 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:33.050464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afd545-3a56-4890-b2e9-e3dfc5ff4631 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:33.050799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afd545-3a56-4890-b2e9-e3dfc5ff4631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:33.087419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-66afd545-3a56-4890-b2e9-e3dfc5ff4631 None None] MFA Rules not processed for user `a05030128ac446fcb09bde4268e56603`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:33.088978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-66afd545-3a56-4890-b2e9-e3dfc5ff4631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:33.187418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 946/1889] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:33 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:33.200243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9f38d-d190-41ad-ba4d-55857ecbb750 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:33.265632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9f38d-d190-41ad-ba4d-55857ecbb750 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:33.270431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b37e07-8ff3-46aa-bed0-bfcc4f4c7440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:33.271478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b37e07-8ff3-46aa-bed0-bfcc4f4c7440 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:33.271478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b37e07-8ff3-46aa-bed0-bfcc4f4c7440 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:33.271657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b37e07-8ff3-46aa-bed0-bfcc4f4c7440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:33.281842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b37e07-8ff3-46aa-bed0-bfcc4f4c7440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:33.283332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b37e07-8ff3-46aa-bed0-bfcc4f4c7440 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:33.293981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 944/1890] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:33 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:33.346241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afd545-3a56-4890-b2e9-e3dfc5ff4631 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:33.347929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-66afd545-3a56-4890-b2e9-e3dfc5ff4631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:33.360245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afd545-3a56-4890-b2e9-e3dfc5ff4631 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:33.364521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac92d5-8ede-4320-b3c8-7ef07330edd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:33.365250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac92d5-8ede-4320-b3c8-7ef07330edd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:33.365597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac92d5-8ede-4320-b3c8-7ef07330edd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:33.365939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac92d5-8ede-4320-b3c8-7ef07330edd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:33.426395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac92d5-8ede-4320-b3c8-7ef07330edd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:33.427888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac92d5-8ede-4320-b3c8-7ef07330edd1 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:33.437839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 947/1891] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:36.043361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b37e07-8ff3-46aa-bed0-bfcc4f4c7440 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:36.052614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b37e07-8ff3-46aa-bed0-bfcc4f4c7440 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:36.057169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30991116-3516-4cc1-ae0b-a4e131423b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:36.057801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991116-3516-4cc1-ae0b-a4e131423b4c service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:36.058088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991116-3516-4cc1-ae0b-a4e131423b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:36.058533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30991116-3516-4cc1-ae0b-a4e131423b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:36.060394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-30991116-3516-4cc1-ae0b-a4e131423b4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:36.129075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30991116-3516-4cc1-ae0b-a4e131423b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:36.130656 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30991116-3516-4cc1-ae0b-a4e131423b4c service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:36.141693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 945/1892] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:37.145115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac92d5-8ede-4320-b3c8-7ef07330edd1 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:37.218245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:37.223733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac92d5-8ede-4320-b3c8-7ef07330edd1 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:37.233979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:37.234666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:37.234951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:37.235226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] PATH_INFO: `/v3/users/39615e87fc7741298f99ab9045b1c969` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:37.236476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39615e87fc7741298f99ab9045b1c969)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:37.238442 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:37.305863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39615e87fc7741298f99ab9045b1c969'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.321945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39615e87fc7741298f99ab9045b1c969'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.332139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39615e87fc7741298f99ab9045b1c969'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.333040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39615e87fc7741298f99ab9045b1c969'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.333813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39615e87fc7741298f99ab9045b1c969'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.356086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 948/1893] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:37 2022] DELETE /identity/v3/users/39615e87fc7741298f99ab9045b1c969 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:37.360885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4293b1b-5de9-4c50-a567-db7257672339 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:37.361430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4293b1b-5de9-4c50-a567-db7257672339 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:37.361430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4293b1b-5de9-4c50-a567-db7257672339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:37.392940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-e4293b1b-5de9-4c50-a567-db7257672339 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:37.394440 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e4293b1b-5de9-4c50-a567-db7257672339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:37.448623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:37.469199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 946/1894] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:37.479789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:37.481410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:37.500971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4293b1b-5de9-4c50-a567-db7257672339 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:37.502944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e4293b1b-5de9-4c50-a567-db7257672339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:37.515458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4293b1b-5de9-4c50-a567-db7257672339 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:37.521052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:37.521707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:37.522141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:37.522414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] PATH_INFO: `/v3/users/4167faace34a428eb582380e4902bd3f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:37.523994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4167faace34a428eb582380e4902bd3f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:37.525992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:37.553904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db539f7-9d46-4d9d-8532-f58aec5a9e58 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:37.558404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee619a1-7fa4-406d-9495-c38c3b35a662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:37.559494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee619a1-7fa4-406d-9495-c38c3b35a662 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:37.559779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee619a1-7fa4-406d-9495-c38c3b35a662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:37.560113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee619a1-7fa4-406d-9495-c38c3b35a662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:37.576396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee619a1-7fa4-406d-9495-c38c3b35a662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:37.576396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee619a1-7fa4-406d-9495-c38c3b35a662 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:37.586029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4167faace34a428eb582380e4902bd3f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.586733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4167faace34a428eb582380e4902bd3f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.591969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 949/1895] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:37.596432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4167faace34a428eb582380e4902bd3f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.611255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4167faace34a428eb582380e4902bd3f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.619042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4167faace34a428eb582380e4902bd3f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.625407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 947/1896] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:37 2022] DELETE /identity/v3/users/4167faace34a428eb582380e4902bd3f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:37.630137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116d968-1421-4764-9c95-764ed25a21b9 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:37.630598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116d968-1421-4764-9c95-764ed25a21b9 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:37.630876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116d968-1421-4764-9c95-764ed25a21b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:37.649944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a116d968-1421-4764-9c95-764ed25a21b9 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:37.651159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a116d968-1421-4764-9c95-764ed25a21b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:37.715058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 950/1897] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:37.725237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:37.726725 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:37.789299 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02817c85-b021-43a3-90a1-dca755ad164f admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:37.793191 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310b265-2964-49c2-a14a-e005bd99656e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:37.793825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310b265-2964-49c2-a14a-e005bd99656e service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:37.794099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310b265-2964-49c2-a14a-e005bd99656e service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:37.794473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310b265-2964-49c2-a14a-e005bd99656e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:37.804445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310b265-2964-49c2-a14a-e005bd99656e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:37.805894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310b265-2964-49c2-a14a-e005bd99656e service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:37.816132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 948/1898] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:37.903172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116d968-1421-4764-9c95-764ed25a21b9 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:37.904742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a116d968-1421-4764-9c95-764ed25a21b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:37.918183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116d968-1421-4764-9c95-764ed25a21b9 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:37.923823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:37.924716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:37.924994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:37.925325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] PATH_INFO: `/v3/projects/d90139165e274a93a72f949e6909294b` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:37.926731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d90139165e274a93a72f949e6909294b)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:37.929009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:37.948557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:37.988521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd90139165e274a93a72f949e6909294b', 'request_id': 'req-426bfef4-9678-4776-aa4b-09421b3c3100'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:37.996174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd90139165e274a93a72f949e6909294b', 'request_id': 'req-426bfef4-9678-4776-aa4b-09421b3c3100'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.005152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd90139165e274a93a72f949e6909294b', 'request_id': 'req-426bfef4-9678-4776-aa4b-09421b3c3100'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.015910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] The token cache is being invalidate because project d90139165e274a93a72f949e6909294b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:38.016196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.016935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.018484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 951/1899] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:37 2022] DELETE /identity/v3/projects/d90139165e274a93a72f949e6909294b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:38.022614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310b265-2964-49c2-a14a-e005bd99656e service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:38.032455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310b265-2964-49c2-a14a-e005bd99656e service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:38.101239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:38.101939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:38.102216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:38.102473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] PATH_INFO: `/v3/users/a05030128ac446fcb09bde4268e56603` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:38.103753 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a05030128ac446fcb09bde4268e56603)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:38.105785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:38.108562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:38.118970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:38.120568 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:38.139196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bfef4-9678-4776-aa4b-09421b3c3100 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:38.144697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:38.145388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:38.146061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:38.146345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] PATH_INFO: `/v3/projects/3ec35db0316b42c4a54563701eb54c5b` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:38.147460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec35db0316b42c4a54563701eb54c5b)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:38.149467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:38.165987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a05030128ac446fcb09bde4268e56603'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.166668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a05030128ac446fcb09bde4268e56603'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.175151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a05030128ac446fcb09bde4268e56603'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.188169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a05030128ac446fcb09bde4268e56603'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.195844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a05030128ac446fcb09bde4268e56603'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.200623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec35db0316b42c4a54563701eb54c5b', 'request_id': 'req-fdcb6fed-b986-41a9-914d-0b73e51c1c22'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.201905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 949/1900] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:38 2022] DELETE /identity/v3/users/a05030128ac446fcb09bde4268e56603 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:38.209618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec35db0316b42c4a54563701eb54c5b', 'request_id': 'req-fdcb6fed-b986-41a9-914d-0b73e51c1c22'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.218462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec35db0316b42c4a54563701eb54c5b', 'request_id': 'req-fdcb6fed-b986-41a9-914d-0b73e51c1c22'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.229527 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] The token cache is being invalidate because project 3ec35db0316b42c4a54563701eb54c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:38.229817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.230478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.232033 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 952/1901] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:38 2022] DELETE /identity/v3/projects/3ec35db0316b42c4a54563701eb54c5b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:38.236211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:38.237685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:38.308593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9d868-e877-46ff-832e-ebbfba791b30 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:38.315468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:38.316164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:38.316557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:38.327122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] PATH_INFO: `/v3/users/8822ef4610924beb90693e7e55412be9` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:38.329421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8822ef4610924beb90693e7e55412be9)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:38.332125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:38.407203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8822ef4610924beb90693e7e55412be9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.408368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8822ef4610924beb90693e7e55412be9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.417977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8822ef4610924beb90693e7e55412be9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.432088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8822ef4610924beb90693e7e55412be9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.439004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8822ef4610924beb90693e7e55412be9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:38.441378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:38.446460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 950/1902] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:38 2022] DELETE /identity/v3/users/8822ef4610924beb90693e7e55412be9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:38.759017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:38.760900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:38.875583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb6fed-b986-41a9-914d-0b73e51c1c22 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:38.923345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:38.967875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:38.968840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:38.969319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:38.969600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] PATH_INFO: `/v3/projects/1b53ce99e71a4e90acd83cfdfd2f4c14` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:38.971034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b53ce99e71a4e90acd83cfdfd2f4c14)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:38.972965 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.046557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.048533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b53ce99e71a4e90acd83cfdfd2f4c14', 'request_id': 'req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:39.048738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:39.056746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b53ce99e71a4e90acd83cfdfd2f4c14', 'request_id': 'req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:39.068151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b53ce99e71a4e90acd83cfdfd2f4c14', 'request_id': 'req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:39.071895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1189a78-91a1-4519-8333-7fe727f47e5a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.081674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] The token cache is being invalidate because project 1b53ce99e71a4e90acd83cfdfd2f4c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:39.081674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:39.082533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ee988ad4-acff-4c0a-bff8-f1e54c07bb65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:39.084363 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 953/1903] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:38 2022] DELETE /identity/v3/projects/1b53ce99e71a4e90acd83cfdfd2f4c14 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:39.089272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fafbd4-eed1-483a-868a-a6eda1fc7712 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.089272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fafbd4-eed1-483a-868a-a6eda1fc7712 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.089478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fafbd4-eed1-483a-868a-a6eda1fc7712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.117563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d1fafbd4-eed1-483a-868a-a6eda1fc7712 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:39.119103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d1fafbd4-eed1-483a-868a-a6eda1fc7712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:39.178388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.179053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.179344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.179609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] PATH_INFO: `/v3/projects/ea390cf3f402462e832de08237f16bc5` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.180801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea390cf3f402462e832de08237f16bc5)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.182774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.187249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 954/1904] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:39.195717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88852425-027d-43c7-bcb5-5f951cb1cdde None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.196028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88852425-027d-43c7-bcb5-5f951cb1cdde None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.196168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88852425-027d-43c7-bcb5-5f951cb1cdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.221212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-88852425-027d-43c7-bcb5-5f951cb1cdde None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:39.222444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-88852425-027d-43c7-bcb5-5f951cb1cdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:39.239187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea390cf3f402462e832de08237f16bc5', 'request_id': 'req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:39.247037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea390cf3f402462e832de08237f16bc5', 'request_id': 'req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:39.258817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea390cf3f402462e832de08237f16bc5', 'request_id': 'req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:39.270817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] The token cache is being invalidate because project ea390cf3f402462e832de08237f16bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:39.271139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:39.271842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4beb0e32-fef2-49be-b0e6-a81fb7d38ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:39.273868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 951/1905] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:39 2022] DELETE /identity/v3/projects/ea390cf3f402462e832de08237f16bc5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:39.278244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2e97d-4696-423d-8305-50b70a985e98 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.278524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2e97d-4696-423d-8305-50b70a985e98 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.278802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2e97d-4696-423d-8305-50b70a985e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.295594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 955/1906] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:39.301142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9bc2e97d-4696-423d-8305-50b70a985e98 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:39.301303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88852425-027d-43c7-bcb5-5f951cb1cdde None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.302442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9bc2e97d-4696-423d-8305-50b70a985e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:39.307275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:39.310816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 952/1907] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:39.311818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88852425-027d-43c7-bcb5-5f951cb1cdde None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.319127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d959d3-d672-4e38-900b-2e1244c498ca None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.319416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d959d3-d672-4e38-900b-2e1244c498ca None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.319691 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d959d3-d672-4e38-900b-2e1244c498ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.321316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5936d28-c7b4-483c-9100-54ec62dcb1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.322190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5936d28-c7b4-483c-9100-54ec62dcb1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.322796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5936d28-c7b4-483c-9100-54ec62dcb1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.323198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5936d28-c7b4-483c-9100-54ec62dcb1ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.324469 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5936d28-c7b4-483c-9100-54ec62dcb1ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:39.325107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5936d28-c7b4-483c-9100-54ec62dcb1ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.328868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5936d28-c7b4-483c-9100-54ec62dcb1ba admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.341944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 956/1908] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:05:39.342989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-78d959d3-d672-4e38-900b-2e1244c498ca None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:39.344205 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-78d959d3-d672-4e38-900b-2e1244c498ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:39.346507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5936d28-c7b4-483c-9100-54ec62dcb1ba admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.352919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 953/1909] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:39.357020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d959d3-d672-4e38-900b-2e1244c498ca None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.357245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5936d28-c7b4-483c-9100-54ec62dcb1ba admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.362176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352831d0-172c-4a25-a521-01496959cb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.363400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352831d0-172c-4a25-a521-01496959cb14 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.363679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352831d0-172c-4a25-a521-01496959cb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.363955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352831d0-172c-4a25-a521-01496959cb14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.364856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352831d0-172c-4a25-a521-01496959cb14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.366479 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d959d3-d672-4e38-900b-2e1244c498ca None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.366780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352831d0-172c-4a25-a521-01496959cb14 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.371688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85a1d2-954d-4c89-9b80-bee880befdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.372307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a1d2-954d-4c89-9b80-bee880befdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.372831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a1d2-954d-4c89-9b80-bee880befdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.372945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85a1d2-954d-4c89-9b80-bee880befdb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.373702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85a1d2-954d-4c89-9b80-bee880befdb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:39.373923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85a1d2-954d-4c89-9b80-bee880befdb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.375443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85a1d2-954d-4c89-9b80-bee880befdb6 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.387865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 954/1910] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:05:39.392579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85a1d2-954d-4c89-9b80-bee880befdb6 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.405368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85a1d2-954d-4c89-9b80-bee880befdb6 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.409371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 957/1911] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:39 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:39.411718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a556f26-7494-450e-b0d0-dfb8e0e4c4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.412574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a556f26-7494-450e-b0d0-dfb8e0e4c4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.412846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a556f26-7494-450e-b0d0-dfb8e0e4c4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.413124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a556f26-7494-450e-b0d0-dfb8e0e4c4e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.413809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a556f26-7494-450e-b0d0-dfb8e0e4c4e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.415980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a556f26-7494-450e-b0d0-dfb8e0e4c4e9 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.425481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352831d0-172c-4a25-a521-01496959cb14 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.452683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 955/1912] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:39 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:39.457394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a556f26-7494-450e-b0d0-dfb8e0e4c4e9 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.505440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352831d0-172c-4a25-a521-01496959cb14 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.521077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:39.553321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a556f26-7494-450e-b0d0-dfb8e0e4c4e9 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.562142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335a652-a448-4810-849e-92c5894353da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.562982 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335a652-a448-4810-849e-92c5894353da admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.563357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335a652-a448-4810-849e-92c5894353da admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.564266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335a652-a448-4810-849e-92c5894353da admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.565096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335a652-a448-4810-849e-92c5894353da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.568263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335a652-a448-4810-849e-92c5894353da admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.595077 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcfa50-ed82-4cb0-bd6e-7111e89e9fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.596125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcfa50-ed82-4cb0-bd6e-7111e89e9fab admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.596605 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcfa50-ed82-4cb0-bd6e-7111e89e9fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.597018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dcfa50-ed82-4cb0-bd6e-7111e89e9fab admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.598211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcfa50-ed82-4cb0-bd6e-7111e89e9fab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.600809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dcfa50-ed82-4cb0-bd6e-7111e89e9fab admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.603790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 956/1913] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:39 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:39.616759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335a652-a448-4810-849e-92c5894353da admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.648406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 958/1914] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:39 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:39.649459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcfa50-ed82-4cb0-bd6e-7111e89e9fab admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.656188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335a652-a448-4810-849e-92c5894353da admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.676238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dcfa50-ed82-4cb0-bd6e-7111e89e9fab admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.688099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32511f2-cfc2-48d3-a5c3-821f867417bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.690452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb4e2d-27bd-4793-b7d1-e922101d0e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.691285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32511f2-cfc2-48d3-a5c3-821f867417bb admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.691571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32511f2-cfc2-48d3-a5c3-821f867417bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.692267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb4e2d-27bd-4793-b7d1-e922101d0e9b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.692785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32511f2-cfc2-48d3-a5c3-821f867417bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.693678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb4e2d-27bd-4793-b7d1-e922101d0e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.693971 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb4e2d-27bd-4793-b7d1-e922101d0e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.694457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32511f2-cfc2-48d3-a5c3-821f867417bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.701199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb4e2d-27bd-4793-b7d1-e922101d0e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.704541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32511f2-cfc2-48d3-a5c3-821f867417bb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.706774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb4e2d-27bd-4793-b7d1-e922101d0e9b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.746008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 959/1915] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:39.747514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 957/1916] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:05:39.753793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32511f2-cfc2-48d3-a5c3-821f867417bb admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.753973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb4e2d-27bd-4793-b7d1-e922101d0e9b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.770764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb4e2d-27bd-4793-b7d1-e922101d0e9b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.780906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be17c8-7403-447f-a86e-43c74b05dad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.781628 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be17c8-7403-447f-a86e-43c74b05dad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.782177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be17c8-7403-447f-a86e-43c74b05dad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.782484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be17c8-7403-447f-a86e-43c74b05dad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.783228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be17c8-7403-447f-a86e-43c74b05dad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.785896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be17c8-7403-447f-a86e-43c74b05dad2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.793460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32511f2-cfc2-48d3-a5c3-821f867417bb admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.808474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e858a-5f03-4246-8909-47bd0414f795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.808474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e858a-5f03-4246-8909-47bd0414f795 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.808474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e858a-5f03-4246-8909-47bd0414f795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.808474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e858a-5f03-4246-8909-47bd0414f795 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.808883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e858a-5f03-4246-8909-47bd0414f795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.809259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 960/1917] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:39.810330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be17c8-7403-447f-a86e-43c74b05dad2 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.813400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e858a-5f03-4246-8909-47bd0414f795 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.824552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 958/1918] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:39.828931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e858a-5f03-4246-8909-47bd0414f795 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:39.840261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be17c8-7403-447f-a86e-43c74b05dad2 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.845643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e858a-5f03-4246-8909-47bd0414f795 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:39.854498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726bf69-0a11-4f40-9051-319454d8dedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.855260 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0f396-cf98-49be-89ae-3433d9b7accd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:39.856248 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0f396-cf98-49be-89ae-3433d9b7accd admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.856501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0f396-cf98-49be-89ae-3433d9b7accd admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.856996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0f396-cf98-49be-89ae-3433d9b7accd admin admin] PATH_INFO: `/v3/projects/8561ccadb0e94f528108c36bfe5f1415/users/01edd8cb1da94a39bcb162e0e292c41e/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.857322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726bf69-0a11-4f40-9051-319454d8dedf admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.857920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726bf69-0a11-4f40-9051-319454d8dedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.858200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726bf69-0a11-4f40-9051-319454d8dedf admin admin] PATH_INFO: `/v3/projects/1bbc1e3041b945319aa331522463318c/users/2af6962653584e16b9f5e35898142794/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.877701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0f396-cf98-49be-89ae-3433d9b7accd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8561ccadb0e94f528108c36bfe5f1415, user_id=01edd8cb1da94a39bcb162e0e292c41e, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.881124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0f396-cf98-49be-89ae-3433d9b7accd admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.887370 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726bf69-0a11-4f40-9051-319454d8dedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbc1e3041b945319aa331522463318c, user_id=2af6962653584e16b9f5e35898142794, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:39.890413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726bf69-0a11-4f40-9051-319454d8dedf admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:39.912010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 961/1919] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:39 2022] PUT /identity/v3/projects/8561ccadb0e94f528108c36bfe5f1415/users/01edd8cb1da94a39bcb162e0e292c41e/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:39.922786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97f45-9156-43c9-b0a6-36c6a1b77a1f None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.923343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97f45-9156-43c9-b0a6-36c6a1b77a1f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.923697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97f45-9156-43c9-b0a6-36c6a1b77a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.931228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 959/1920] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:39 2022] PUT /identity/v3/projects/1bbc1e3041b945319aa331522463318c/users/2af6962653584e16b9f5e35898142794/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:39.943278 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c509a-472d-4577-9054-fa82ecf23316 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:39.943953 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c509a-472d-4577-9054-fa82ecf23316 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:39.944491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c509a-472d-4577-9054-fa82ecf23316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:39.959339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-dcf97f45-9156-43c9-b0a6-36c6a1b77a1f None None] MFA Rules not processed for user `01edd8cb1da94a39bcb162e0e292c41e`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:39.961031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-dcf97f45-9156-43c9-b0a6-36c6a1b77a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:39.971107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:40.012373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-b63c509a-472d-4577-9054-fa82ecf23316 None None] MFA Rules not processed for user `2af6962653584e16b9f5e35898142794`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:40.013907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b63c509a-472d-4577-9054-fa82ecf23316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:40.079731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 962/1921] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:40.085601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf97f45-9156-43c9-b0a6-36c6a1b77a1f None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.133129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 960/1922] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:40.137482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c509a-472d-4577-9054-fa82ecf23316 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.212138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf97f45-9156-43c9-b0a6-36c6a1b77a1f None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.217142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6374e-3dcd-484f-a019-a5648c39b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.218172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6374e-3dcd-484f-a019-a5648c39b452 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.218453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6374e-3dcd-484f-a019-a5648c39b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.218872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6374e-3dcd-484f-a019-a5648c39b452 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.220616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6374e-3dcd-484f-a019-a5648c39b452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.223499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6374e-3dcd-484f-a019-a5648c39b452 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.225273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c509a-472d-4577-9054-fa82ecf23316 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.238284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafd638-9de1-4dc7-8c54-f8252e973816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.239100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafd638-9de1-4dc7-8c54-f8252e973816 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.239384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafd638-9de1-4dc7-8c54-f8252e973816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.239956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafd638-9de1-4dc7-8c54-f8252e973816 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.240561 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafd638-9de1-4dc7-8c54-f8252e973816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.245141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafd638-9de1-4dc7-8c54-f8252e973816 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.265344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 963/1923] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:40 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:40.269005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6374e-3dcd-484f-a019-a5648c39b452 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.313605 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:40.330442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 961/1924] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:40 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:40.331783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafd638-9de1-4dc7-8c54-f8252e973816 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.333765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bfafd638-9de1-4dc7-8c54-f8252e973816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:40.374686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6374e-3dcd-484f-a019-a5648c39b452 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.432626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:40.457952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502daee-823e-4fe7-b2b0-5c08c5b42b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.458838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502daee-823e-4fe7-b2b0-5c08c5b42b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.459212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502daee-823e-4fe7-b2b0-5c08c5b42b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.460238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502daee-823e-4fe7-b2b0-5c08c5b42b7e admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.460603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafd638-9de1-4dc7-8c54-f8252e973816 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.461266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502daee-823e-4fe7-b2b0-5c08c5b42b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.463678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502daee-823e-4fe7-b2b0-5c08c5b42b7e admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.467845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.468789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.469246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.469692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] PATH_INFO: `/v3/users/4784dd7e7e364c7c808b9ce17c9ab409` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.471378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4784dd7e7e364c7c808b9ce17c9ab409)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.473735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.499809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 964/1925] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:40 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:40.500824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502daee-823e-4fe7-b2b0-5c08c5b42b7e admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.521317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502daee-823e-4fe7-b2b0-5c08c5b42b7e admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.529455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41921a0-e1e6-48dc-86ff-ee7ed998c6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.529897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41921a0-e1e6-48dc-86ff-ee7ed998c6fe admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.530352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41921a0-e1e6-48dc-86ff-ee7ed998c6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.530880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41921a0-e1e6-48dc-86ff-ee7ed998c6fe admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.531828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41921a0-e1e6-48dc-86ff-ee7ed998c6fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.535548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41921a0-e1e6-48dc-86ff-ee7ed998c6fe admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.552910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4784dd7e7e364c7c808b9ce17c9ab409'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:40.553810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4784dd7e7e364c7c808b9ce17c9ab409'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:40.562065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 965/1926] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:40 2022] POST /identity/v3/users => generated 289 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:40.563183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41921a0-e1e6-48dc-86ff-ee7ed998c6fe admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.564314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4784dd7e7e364c7c808b9ce17c9ab409'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:40.565620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b41921a0-e1e6-48dc-86ff-ee7ed998c6fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:40.579836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41921a0-e1e6-48dc-86ff-ee7ed998c6fe admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.581644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4784dd7e7e364c7c808b9ce17c9ab409'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:40.585952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811b6df-b9cb-453d-9e11-d015223be931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.586648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811b6df-b9cb-453d-9e11-d015223be931 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.586921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811b6df-b9cb-453d-9e11-d015223be931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.587195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811b6df-b9cb-453d-9e11-d015223be931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.587900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811b6df-b9cb-453d-9e11-d015223be931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.589714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811b6df-b9cb-453d-9e11-d015223be931 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.590084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4784dd7e7e364c7c808b9ce17c9ab409'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:40.601119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 962/1927] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:40 2022] DELETE /identity/v3/users/4784dd7e7e364c7c808b9ce17c9ab409 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:40.602001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.603365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:40.605923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 966/1928] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:40.607232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479973e-4276-4737-9d6f-6a3f9a4f0fd1 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.607518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479973e-4276-4737-9d6f-6a3f9a4f0fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.607806 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479973e-4276-4737-9d6f-6a3f9a4f0fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.629308 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-8479973e-4276-4737-9d6f-6a3f9a4f0fd1 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:40.630569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8479973e-4276-4737-9d6f-6a3f9a4f0fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:40.675579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938323a-06c0-4ee2-957e-9698ef2bbdcc admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.680695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf67be4-29bc-41a4-b299-c3afe2a9fb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.681364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf67be4-29bc-41a4-b299-c3afe2a9fb41 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.681646 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf67be4-29bc-41a4-b299-c3afe2a9fb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.681921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf67be4-29bc-41a4-b299-c3afe2a9fb41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.682625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf67be4-29bc-41a4-b299-c3afe2a9fb41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.684320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf67be4-29bc-41a4-b299-c3afe2a9fb41 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.689799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 967/1929] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:40.690923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479973e-4276-4737-9d6f-6a3f9a4f0fd1 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.694955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 963/1930] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:40.698828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf67be4-29bc-41a4-b299-c3afe2a9fb41 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.708758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479973e-4276-4737-9d6f-6a3f9a4f0fd1 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.709445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf67be4-29bc-41a4-b299-c3afe2a9fb41 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.713823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29731cac-bd81-4fb8-8446-6e189e9fb08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.714450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29731cac-bd81-4fb8-8446-6e189e9fb08c admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.714727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29731cac-bd81-4fb8-8446-6e189e9fb08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.714992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29731cac-bd81-4fb8-8446-6e189e9fb08c admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.715177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ef8c3-5c42-47c4-93ee-e94ca287fe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.715692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29731cac-bd81-4fb8-8446-6e189e9fb08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.716028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ef8c3-5c42-47c4-93ee-e94ca287fe6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.716411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ef8c3-5c42-47c4-93ee-e94ca287fe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.716741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ef8c3-5c42-47c4-93ee-e94ca287fe6d admin admin] PATH_INFO: `/v3/projects/e5d9fb09f1a947e5a353bf5ed5b149cb/users/b8bb42fd2a2b45238e2d407540f6f295/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.717339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29731cac-bd81-4fb8-8446-6e189e9fb08c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.727761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 968/1931] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:40.728626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29731cac-bd81-4fb8-8446-6e189e9fb08c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.730597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-29731cac-bd81-4fb8-8446-6e189e9fb08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:40.732233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:40.736126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ef8c3-5c42-47c4-93ee-e94ca287fe6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d9fb09f1a947e5a353bf5ed5b149cb, user_id=b8bb42fd2a2b45238e2d407540f6f295, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.739428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ef8c3-5c42-47c4-93ee-e94ca287fe6d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.755865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 964/1932] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:40 2022] PUT /identity/v3/projects/e5d9fb09f1a947e5a353bf5ed5b149cb/users/b8bb42fd2a2b45238e2d407540f6f295/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:40.757168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ef8c3-5c42-47c4-93ee-e94ca287fe6d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.790041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29731cac-bd81-4fb8-8446-6e189e9fb08c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.823962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ef8c3-5c42-47c4-93ee-e94ca287fe6d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.833878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458ecfa-c0a3-45ab-984d-680c504c8a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.834584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458ecfa-c0a3-45ab-984d-680c504c8a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.834864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458ecfa-c0a3-45ab-984d-680c504c8a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.835145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458ecfa-c0a3-45ab-984d-680c504c8a7a admin admin] PATH_INFO: `/v3/projects/5afb7433d0f24444bb69fdd50f56c0c3/users/c35f0efeb0774e6fb5eebb7ca29ac83d/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.835560 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:40.839940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950019e-a030-4de3-ad4f-866f5b702f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.840659 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950019e-a030-4de3-ad4f-866f5b702f1a service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.841179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950019e-a030-4de3-ad4f-866f5b702f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.841446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950019e-a030-4de3-ad4f-866f5b702f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.852432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950019e-a030-4de3-ad4f-866f5b702f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.854950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a950019e-a030-4de3-ad4f-866f5b702f1a service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.856209 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458ecfa-c0a3-45ab-984d-680c504c8a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afb7433d0f24444bb69fdd50f56c0c3, user_id=c35f0efeb0774e6fb5eebb7ca29ac83d, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.858934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458ecfa-c0a3-45ab-984d-680c504c8a7a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.865724 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 969/1933] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:40.866685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950019e-a030-4de3-ad4f-866f5b702f1a service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.874190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 965/1934] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:40 2022] PUT /identity/v3/projects/5afb7433d0f24444bb69fdd50f56c0c3/users/c35f0efeb0774e6fb5eebb7ca29ac83d/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:40.884716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982d48f-3133-48f4-873b-c41faa815e2b None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.885001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982d48f-3133-48f4-873b-c41faa815e2b None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.885307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982d48f-3133-48f4-873b-c41faa815e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.886261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a950019e-a030-4de3-ad4f-866f5b702f1a service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.921021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a982d48f-3133-48f4-873b-c41faa815e2b None None] MFA Rules not processed for user `c35f0efeb0774e6fb5eebb7ca29ac83d`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:40.922474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a982d48f-3133-48f4-873b-c41faa815e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:40.957503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92801867-d5ef-44ff-a197-b4a129896bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.958211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92801867-d5ef-44ff-a197-b4a129896bf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.958534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92801867-d5ef-44ff-a197-b4a129896bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.958869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92801867-d5ef-44ff-a197-b4a129896bf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.959575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92801867-d5ef-44ff-a197-b4a129896bf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.961222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92801867-d5ef-44ff-a197-b4a129896bf5 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:40.972063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 970/1935] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:40.976616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92801867-d5ef-44ff-a197-b4a129896bf5 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:40.987293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92801867-d5ef-44ff-a197-b4a129896bf5 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:40.992394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00f5a6-bc88-459f-9b47-e46f9d9f9a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:40.993080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00f5a6-bc88-459f-9b47-e46f9d9f9a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:40.993719 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00f5a6-bc88-459f-9b47-e46f9d9f9a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:40.994003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00f5a6-bc88-459f-9b47-e46f9d9f9a20 admin admin] PATH_INFO: `/v3/projects/e5d9fb09f1a947e5a353bf5ed5b149cb/users/b8bb42fd2a2b45238e2d407540f6f295/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:40.995998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00f5a6-bc88-459f-9b47-e46f9d9f9a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d9fb09f1a947e5a353bf5ed5b149cb, user_id=b8bb42fd2a2b45238e2d407540f6f295, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:40.998734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00f5a6-bc88-459f-9b47-e46f9d9f9a20 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:41.005669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 966/1936] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:41.010505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982d48f-3133-48f4-873b-c41faa815e2b None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:41.032379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 971/1937] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:40 2022] PUT /identity/v3/projects/e5d9fb09f1a947e5a353bf5ed5b149cb/users/b8bb42fd2a2b45238e2d407540f6f295/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:41.042275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304db055-8d2e-44cc-9fac-ce986aaeb13b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:41.042641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304db055-8d2e-44cc-9fac-ce986aaeb13b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:41.042995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304db055-8d2e-44cc-9fac-ce986aaeb13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:41.048310 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982d48f-3133-48f4-873b-c41faa815e2b None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:41.077610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-304db055-8d2e-44cc-9fac-ce986aaeb13b None None] MFA Rules not processed for user `b8bb42fd2a2b45238e2d407540f6f295`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:41.078873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-304db055-8d2e-44cc-9fac-ce986aaeb13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:41.150188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25101a1-53f0-4e34-b7c3-faa41c39f6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:41.151171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25101a1-53f0-4e34-b7c3-faa41c39f6ad admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:41.151558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25101a1-53f0-4e34-b7c3-faa41c39f6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:41.151927 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25101a1-53f0-4e34-b7c3-faa41c39f6ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:41.152746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25101a1-53f0-4e34-b7c3-faa41c39f6ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:41.155083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25101a1-53f0-4e34-b7c3-faa41c39f6ad admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:41.195454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:41.212198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 972/1938] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:41.225726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304db055-8d2e-44cc-9fac-ce986aaeb13b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:41.228673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-304db055-8d2e-44cc-9fac-ce986aaeb13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:41.239685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 967/1939] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:41 2022] POST /identity/v3/projects => generated 387 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:41.240890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25101a1-53f0-4e34-b7c3-faa41c39f6ad admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:41.354039 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304db055-8d2e-44cc-9fac-ce986aaeb13b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:41.362273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25101a1-53f0-4e34-b7c3-faa41c39f6ad admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:41.370303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe946ce-694c-4420-a883-065da0533844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:41.370968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe946ce-694c-4420-a883-065da0533844 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:41.371284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe946ce-694c-4420-a883-065da0533844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:41.371565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe946ce-694c-4420-a883-065da0533844 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:41.372265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe946ce-694c-4420-a883-065da0533844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:41.374393 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe946ce-694c-4420-a883-065da0533844 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:41.375332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:41.421664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 968/1940] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:41 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:41.421664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe946ce-694c-4420-a883-065da0533844 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:41.428743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bbe946ce-694c-4420-a883-065da0533844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:41.447817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe946ce-694c-4420-a883-065da0533844 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:41.458953 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:41.460575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:41.461167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:41.462723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] PATH_INFO: `/v3/projects/e2f831065d8c4b27b77e8bfbaf50f7f3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:41.465287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f831065d8c4b27b77e8bfbaf50f7f3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:41.468479 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:41.488978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab44111-5780-4b5b-b73b-1a102012dc07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:41.489875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab44111-5780-4b5b-b73b-1a102012dc07 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:41.490453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab44111-5780-4b5b-b73b-1a102012dc07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:41.490563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab44111-5780-4b5b-b73b-1a102012dc07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:41.543277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f831065d8c4b27b77e8bfbaf50f7f3', 'request_id': 'req-345ac0e9-5e1f-49e4-8226-039de8cd42c1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:41.552974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f831065d8c4b27b77e8bfbaf50f7f3', 'request_id': 'req-345ac0e9-5e1f-49e4-8226-039de8cd42c1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:41.565751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f831065d8c4b27b77e8bfbaf50f7f3', 'request_id': 'req-345ac0e9-5e1f-49e4-8226-039de8cd42c1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:41.574928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab44111-5780-4b5b-b73b-1a102012dc07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:41.576913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab44111-5780-4b5b-b73b-1a102012dc07 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:41.586308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] The token cache is being invalidate because project e2f831065d8c4b27b77e8bfbaf50f7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:41.587013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:41.588158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:41.592954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 969/1941] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:41 2022] DELETE /identity/v3/projects/e2f831065d8c4b27b77e8bfbaf50f7f3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:41.592954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:41.593632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:41.599442 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 973/1942] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:41.600501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab44111-5780-4b5b-b73b-1a102012dc07 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:41.630897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab44111-5780-4b5b-b73b-1a102012dc07 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:41.718683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ac0e9-5e1f-49e4-8226-039de8cd42c1 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:41.724333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58685814-9427-4d23-bbb7-bb1a2c238803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:41.725007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58685814-9427-4d23-bbb7-bb1a2c238803 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:41.725287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58685814-9427-4d23-bbb7-bb1a2c238803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:41.725604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58685814-9427-4d23-bbb7-bb1a2c238803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:41.726320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58685814-9427-4d23-bbb7-bb1a2c238803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:41.727971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58685814-9427-4d23-bbb7-bb1a2c238803 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:41.730629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:41.733758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:41.734803 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:41.735247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:41.735536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] PATH_INFO: `/v3/users/ea7133129e7342b786dd3f5539bd0318` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:41.737673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea7133129e7342b786dd3f5539bd0318)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:41.740389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:41.740614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 970/1943] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:41.748248 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58685814-9427-4d23-bbb7-bb1a2c238803 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:41.752260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-58685814-9427-4d23-bbb7-bb1a2c238803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:41.809086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea7133129e7342b786dd3f5539bd0318'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:41.823189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58685814-9427-4d23-bbb7-bb1a2c238803 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:41.826186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea7133129e7342b786dd3f5539bd0318'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:41.827163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36606959-984b-4b5d-80ca-797f903e2433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:41.827831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36606959-984b-4b5d-80ca-797f903e2433 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:41.828316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36606959-984b-4b5d-80ca-797f903e2433 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:41.828821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36606959-984b-4b5d-80ca-797f903e2433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:41.831861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-36606959-984b-4b5d-80ca-797f903e2433 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:41.842023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:41.857994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea7133129e7342b786dd3f5539bd0318'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:41.859693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea7133129e7342b786dd3f5539bd0318'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:41.860561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea7133129e7342b786dd3f5539bd0318'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:41.880722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 974/1944] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:41 2022] DELETE /identity/v3/users/ea7133129e7342b786dd3f5539bd0318 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:41.881676 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:41.883185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:41.953288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36606959-984b-4b5d-80ca-797f903e2433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:41.957314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36606959-984b-4b5d-80ca-797f903e2433 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:41.974384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 971/1945] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:41.974551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2114e-99e2-4120-9af2-8ffc50bb26d8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:41.979675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4406f4-9e10-44e6-b7fa-5a2d7b88e2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:41.980552 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4406f4-9e10-44e6-b7fa-5a2d7b88e2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:41.980900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4406f4-9e10-44e6-b7fa-5a2d7b88e2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:41.981265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4406f4-9e10-44e6-b7fa-5a2d7b88e2bb admin admin] PATH_INFO: `/v3/projects/93845215c0e541cc8180df404bdbada6/users/3fc1ad0f5c8643488da7eb9b6e90b431/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:42.002015 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4406f4-9e10-44e6-b7fa-5a2d7b88e2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=93845215c0e541cc8180df404bdbada6, user_id=3fc1ad0f5c8643488da7eb9b6e90b431, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:42.004996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4406f4-9e10-44e6-b7fa-5a2d7b88e2bb admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:42.040343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 975/1946] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:41 2022] PUT /identity/v3/projects/93845215c0e541cc8180df404bdbada6/users/3fc1ad0f5c8643488da7eb9b6e90b431/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:42.044732 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36606959-984b-4b5d-80ca-797f903e2433 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:42.066466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36606959-984b-4b5d-80ca-797f903e2433 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:42.149074 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ea0af-28cc-4870-bce0-4a1ed45060af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:42.149753 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ea0af-28cc-4870-bce0-4a1ed45060af admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:42.150331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ea0af-28cc-4870-bce0-4a1ed45060af admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:42.150638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ea0af-28cc-4870-bce0-4a1ed45060af admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:42.151882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ea0af-28cc-4870-bce0-4a1ed45060af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:42.153593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ea0af-28cc-4870-bce0-4a1ed45060af admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:42.177153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 972/1947] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:42.178132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4406f4-9e10-44e6-b7fa-5a2d7b88e2bb admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:42.193552 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4406f4-9e10-44e6-b7fa-5a2d7b88e2bb admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:42.215184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26e3df-666d-4d59-b5ff-83c6a1715e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:42.215184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26e3df-666d-4d59-b5ff-83c6a1715e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:42.215184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26e3df-666d-4d59-b5ff-83c6a1715e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:42.215184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26e3df-666d-4d59-b5ff-83c6a1715e2f admin admin] PATH_INFO: `/v3/projects/93845215c0e541cc8180df404bdbada6/users/3fc1ad0f5c8643488da7eb9b6e90b431/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:42.219015 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26e3df-666d-4d59-b5ff-83c6a1715e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=93845215c0e541cc8180df404bdbada6, user_id=3fc1ad0f5c8643488da7eb9b6e90b431, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:42.222276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26e3df-666d-4d59-b5ff-83c6a1715e2f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:42.241225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 976/1948] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:42 2022] PUT /identity/v3/projects/93845215c0e541cc8180df404bdbada6/users/3fc1ad0f5c8643488da7eb9b6e90b431/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:42.253032 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602de1-a9ed-4cae-b0e1-72f7f93acb5a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:42.253513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602de1-a9ed-4cae-b0e1-72f7f93acb5a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:42.253963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602de1-a9ed-4cae-b0e1-72f7f93acb5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:42.296339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-33602de1-a9ed-4cae-b0e1-72f7f93acb5a None None] MFA Rules not processed for user `3fc1ad0f5c8643488da7eb9b6e90b431`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:42.296339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-33602de1-a9ed-4cae-b0e1-72f7f93acb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:42.345647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:42.348209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26e3df-666d-4d59-b5ff-83c6a1715e2f admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:42.350284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0d26e3df-666d-4d59-b5ff-83c6a1715e2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:42.383110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 973/1949] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:42.393433 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602de1-a9ed-4cae-b0e1-72f7f93acb5a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:42.394979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-33602de1-a9ed-4cae-b0e1-72f7f93acb5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:42.430704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26e3df-666d-4d59-b5ff-83c6a1715e2f admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:42.435705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:42.436674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:42.436674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:42.436932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] PATH_INFO: `/v3/projects/53ae198cf5ec477ea68baacfa29c29a5` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:42.438579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ae198cf5ec477ea68baacfa29c29a5)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:42.440051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:42.463293 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602de1-a9ed-4cae-b0e1-72f7f93acb5a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:42.468117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384632f1-0bdd-4ae7-9018-413181d251f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:42.469105 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384632f1-0bdd-4ae7-9018-413181d251f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:42.469409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384632f1-0bdd-4ae7-9018-413181d251f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:42.469970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384632f1-0bdd-4ae7-9018-413181d251f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:42.482022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384632f1-0bdd-4ae7-9018-413181d251f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:42.483556 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384632f1-0bdd-4ae7-9018-413181d251f2 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:42.489410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ae198cf5ec477ea68baacfa29c29a5', 'request_id': 'req-1503586b-205d-45aa-addd-bcfe8bd0b524'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:42.495702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 974/1950] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:05:42.496657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ae198cf5ec477ea68baacfa29c29a5', 'request_id': 'req-1503586b-205d-45aa-addd-bcfe8bd0b524'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:42.499799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:42.506512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ae198cf5ec477ea68baacfa29c29a5', 'request_id': 'req-1503586b-205d-45aa-addd-bcfe8bd0b524'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:42.519583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] The token cache is being invalidate because project 53ae198cf5ec477ea68baacfa29c29a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:42.519968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:42.520715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:42.522265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 977/1951] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:42 2022] DELETE /identity/v3/projects/53ae198cf5ec477ea68baacfa29c29a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:42.527366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384632f1-0bdd-4ae7-9018-413181d251f2 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:42.540848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384632f1-0bdd-4ae7-9018-413181d251f2 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:42.607044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:42.607703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:42.607991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:42.608265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] PATH_INFO: `/v3/users/4a05704980a844909dfdb9bad0c44dd0` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:42.609579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a05704980a844909dfdb9bad0c44dd0)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:42.611525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:42.660452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a05704980a844909dfdb9bad0c44dd0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:42.661163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a05704980a844909dfdb9bad0c44dd0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:42.675116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a05704980a844909dfdb9bad0c44dd0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:42.688830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a05704980a844909dfdb9bad0c44dd0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:42.695226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:42.697457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a05704980a844909dfdb9bad0c44dd0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:42.705668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 975/1952] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:42 2022] DELETE /identity/v3/users/4a05704980a844909dfdb9bad0c44dd0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:43.005726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:43.008850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:43.043940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503586b-205d-45aa-addd-bcfe8bd0b524 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:43.086554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:43.089009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:43.117180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:43.118255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.119110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.119576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] PATH_INFO: `/v3/projects/8490b44172be4c98b057e3dab45a04a3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.120961 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8490b44172be4c98b057e3dab45a04a3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:43.123275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:43.164099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3fd1ea-8362-4faa-9f90-ad790bf1bc71 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:43.185859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8490b44172be4c98b057e3dab45a04a3', 'request_id': 'req-f8ee0196-febb-421d-99a4-75d1a175d98b'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:43.192998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8490b44172be4c98b057e3dab45a04a3', 'request_id': 'req-f8ee0196-febb-421d-99a4-75d1a175d98b'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:43.203310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8490b44172be4c98b057e3dab45a04a3', 'request_id': 'req-f8ee0196-febb-421d-99a4-75d1a175d98b'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:43.218003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] The token cache is being invalidate because project 8490b44172be4c98b057e3dab45a04a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:43.219030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:43.219781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-f8ee0196-febb-421d-99a4-75d1a175d98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:43.221460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 978/1953] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:43 2022] DELETE /identity/v3/projects/8490b44172be4c98b057e3dab45a04a3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:43.221673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcade8e-dff5-4f03-a736-21b0f6560e67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:43.222364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcade8e-dff5-4f03-a736-21b0f6560e67 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.222715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcade8e-dff5-4f03-a736-21b0f6560e67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.223023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcade8e-dff5-4f03-a736-21b0f6560e67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.225012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-cbcade8e-dff5-4f03-a736-21b0f6560e67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:43.227176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf56e0-6612-4cd4-afc0-c3dfea9a26ed None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.227461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf56e0-6612-4cd4-afc0-c3dfea9a26ed None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.227736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf56e0-6612-4cd4-afc0-c3dfea9a26ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.248147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-feaf56e0-6612-4cd4-afc0-c3dfea9a26ed None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:43.249479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-feaf56e0-6612-4cd4-afc0-c3dfea9a26ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:43.318846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcade8e-dff5-4f03-a736-21b0f6560e67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:43.319118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcade8e-dff5-4f03-a736-21b0f6560e67 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:43.336060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 976/1954] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:05:43.361808 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 979/1955] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:43.372234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e61119-065e-43a2-a4d9-8881dd4ed0fd None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.372697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e61119-065e-43a2-a4d9-8881dd4ed0fd None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.372974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e61119-065e-43a2-a4d9-8881dd4ed0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.414274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-36e61119-065e-43a2-a4d9-8881dd4ed0fd None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:43.415553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-36e61119-065e-43a2-a4d9-8881dd4ed0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:43.426433 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 977/1956] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:43.430927 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf56e0-6612-4cd4-afc0-c3dfea9a26ed None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:43.442756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf56e0-6612-4cd4-afc0-c3dfea9a26ed None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:43.457960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ff8e8-8830-4bcc-a2ed-01b5c4167eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:43.459091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ff8e8-8830-4bcc-a2ed-01b5c4167eaf admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.459677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ff8e8-8830-4bcc-a2ed-01b5c4167eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.460168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87ff8e8-8830-4bcc-a2ed-01b5c4167eaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.461100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ff8e8-8830-4bcc-a2ed-01b5c4167eaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:43.462388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ff8e8-8830-4bcc-a2ed-01b5c4167eaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:43.467412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87ff8e8-8830-4bcc-a2ed-01b5c4167eaf admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:43.489487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 980/1957] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:05:43.497642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e61119-065e-43a2-a4d9-8881dd4ed0fd None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:43.510826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e61119-065e-43a2-a4d9-8881dd4ed0fd None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:43.519073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a52bb-3455-4714-ac50-d4a8f3e57204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:43.520056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a52bb-3455-4714-ac50-d4a8f3e57204 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.520834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a52bb-3455-4714-ac50-d4a8f3e57204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.521301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a52bb-3455-4714-ac50-d4a8f3e57204 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.522652 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a52bb-3455-4714-ac50-d4a8f3e57204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:43.525554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a52bb-3455-4714-ac50-d4a8f3e57204 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:43.568040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 978/1958] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:43 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:43.569960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ff8e8-8830-4bcc-a2ed-01b5c4167eaf admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:43.575658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:43.585751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87ff8e8-8830-4bcc-a2ed-01b5c4167eaf admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:43.719020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650adcc-450d-411b-ba96-0558036e7e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:43.720084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650adcc-450d-411b-ba96-0558036e7e20 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.721936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650adcc-450d-411b-ba96-0558036e7e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.722596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650adcc-450d-411b-ba96-0558036e7e20 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.723905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650adcc-450d-411b-ba96-0558036e7e20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:43.726731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650adcc-450d-411b-ba96-0558036e7e20 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:43.766820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 981/1959] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:43 2022] POST /identity/v3/users => generated 291 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:43.770315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a52bb-3455-4714-ac50-d4a8f3e57204 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:43.786645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a52bb-3455-4714-ac50-d4a8f3e57204 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:43.795293 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832cf7e-29a9-4ffb-aa26-d54245ad7bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:43.796742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832cf7e-29a9-4ffb-aa26-d54245ad7bba admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.797121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832cf7e-29a9-4ffb-aa26-d54245ad7bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.797487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b832cf7e-29a9-4ffb-aa26-d54245ad7bba admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.798389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832cf7e-29a9-4ffb-aa26-d54245ad7bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:43.801056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b832cf7e-29a9-4ffb-aa26-d54245ad7bba admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:43.816354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 979/1960] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:43.819425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650adcc-450d-411b-ba96-0558036e7e20 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:43.833555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650adcc-450d-411b-ba96-0558036e7e20 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:43.842362 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbf097-d85e-41dc-ac98-2ac4d95daf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:43.842863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbf097-d85e-41dc-ac98-2ac4d95daf67 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.843146 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbf097-d85e-41dc-ac98-2ac4d95daf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.843785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbf097-d85e-41dc-ac98-2ac4d95daf67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.844938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbf097-d85e-41dc-ac98-2ac4d95daf67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:43.846380 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbf097-d85e-41dc-ac98-2ac4d95daf67 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:43.863489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 982/1961] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:43.864679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832cf7e-29a9-4ffb-aa26-d54245ad7bba admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:43.881608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b832cf7e-29a9-4ffb-aa26-d54245ad7bba admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:43.888463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d8774-684a-4f2f-90e8-5576a8318092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:43.888814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8774-684a-4f2f-90e8-5576a8318092 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.889155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8774-684a-4f2f-90e8-5576a8318092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.889490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d8774-684a-4f2f-90e8-5576a8318092 admin admin] PATH_INFO: `/v3/projects/7c4d9bb518704c39a86f8c50265bbeb5/users/e72b440d371a40a589dc00b1ffaf2fda/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.911983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d8774-684a-4f2f-90e8-5576a8318092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c4d9bb518704c39a86f8c50265bbeb5, user_id=e72b440d371a40a589dc00b1ffaf2fda, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:43.915564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d8774-684a-4f2f-90e8-5576a8318092 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:43.935713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 980/1962] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:43 2022] PUT /identity/v3/projects/7c4d9bb518704c39a86f8c50265bbeb5/users/e72b440d371a40a589dc00b1ffaf2fda/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:43.947333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9f698-9a91-4567-9731-d550310410af None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:43.947619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9f698-9a91-4567-9731-d550310410af None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:43.947890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9f698-9a91-4567-9731-d550310410af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:43.977143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d5a9f698-9a91-4567-9731-d550310410af None None] MFA Rules not processed for user `e72b440d371a40a589dc00b1ffaf2fda`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:43.978379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d5a9f698-9a91-4567-9731-d550310410af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:44.049444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 983/1963] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:44.054154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d8774-684a-4f2f-90e8-5576a8318092 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:44.058935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:44.122914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d8774-684a-4f2f-90e8-5576a8318092 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:44.129428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7624c-36ec-4551-8bb2-2b44c598ea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:44.130163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7624c-36ec-4551-8bb2-2b44c598ea42 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.130447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7624c-36ec-4551-8bb2-2b44c598ea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.130723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab7624c-36ec-4551-8bb2-2b44c598ea42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.131383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7624c-36ec-4551-8bb2-2b44c598ea42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:44.133351 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab7624c-36ec-4551-8bb2-2b44c598ea42 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:44.163380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 981/1964] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:44 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:44.169278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9f698-9a91-4567-9731-d550310410af None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:44.180937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9f698-9a91-4567-9731-d550310410af None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:44.268426 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818db62-7f2d-422a-8e1c-16b72d03a7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:44.269098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818db62-7f2d-422a-8e1c-16b72d03a7af admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.269348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818db62-7f2d-422a-8e1c-16b72d03a7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.269622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818db62-7f2d-422a-8e1c-16b72d03a7af admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.270572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818db62-7f2d-422a-8e1c-16b72d03a7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:44.272419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818db62-7f2d-422a-8e1c-16b72d03a7af admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:44.292964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:44.303276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7624c-36ec-4551-8bb2-2b44c598ea42 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:44.303734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 984/1965] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:44 2022] POST /identity/v3/users => generated 297 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:44.317957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab7624c-36ec-4551-8bb2-2b44c598ea42 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:44.324598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296b20d-af2a-417a-b73f-3d33f2e8af99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:44.325272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296b20d-af2a-417a-b73f-3d33f2e8af99 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.325574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296b20d-af2a-417a-b73f-3d33f2e8af99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.325824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296b20d-af2a-417a-b73f-3d33f2e8af99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.326572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296b20d-af2a-417a-b73f-3d33f2e8af99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:44.328394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296b20d-af2a-417a-b73f-3d33f2e8af99 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:44.340022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 982/1966] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:05:44.344404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818db62-7f2d-422a-8e1c-16b72d03a7af admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:44.363811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818db62-7f2d-422a-8e1c-16b72d03a7af admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:44.371219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67f216-776c-4bf1-8999-d8ac1110fd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:44.372466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f216-776c-4bf1-8999-d8ac1110fd6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.372814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f216-776c-4bf1-8999-d8ac1110fd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.373962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f216-776c-4bf1-8999-d8ac1110fd6f admin admin] PATH_INFO: `/v3/projects/3bfa756694494ed799b2d7a18f75edfd/users/71a07c10a77e4817b07701888a45b109/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.401811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67f216-776c-4bf1-8999-d8ac1110fd6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bfa756694494ed799b2d7a18f75edfd, user_id=71a07c10a77e4817b07701888a45b109, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:44.408439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67f216-776c-4bf1-8999-d8ac1110fd6f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:44.426845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 985/1967] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:44 2022] PUT /identity/v3/projects/3bfa756694494ed799b2d7a18f75edfd/users/71a07c10a77e4817b07701888a45b109/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:44.431058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296b20d-af2a-417a-b73f-3d33f2e8af99 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:44.441971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296b20d-af2a-417a-b73f-3d33f2e8af99 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:44.457834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be1556-ab0c-43b7-b4ec-aedd6cb0d12c None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.459016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be1556-ab0c-43b7-b4ec-aedd6cb0d12c None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.459587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be1556-ab0c-43b7-b4ec-aedd6cb0d12c None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.462710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 986/1968] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:05:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 12:05:44.481853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14751194-608a-4478-95d8-cfdf4c14dbdb None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.482343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14751194-608a-4478-95d8-cfdf4c14dbdb None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.482631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14751194-608a-4478-95d8-cfdf4c14dbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.512021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-14751194-608a-4478-95d8-cfdf4c14dbdb None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:44.513623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-14751194-608a-4478-95d8-cfdf4c14dbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:44.515676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a3aa2-a063-4ffb-9318-ee77066079a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:44.516357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a3aa2-a063-4ffb-9318-ee77066079a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.516746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a3aa2-a063-4ffb-9318-ee77066079a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.517120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a3aa2-a063-4ffb-9318-ee77066079a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.517977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a3aa2-a063-4ffb-9318-ee77066079a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:44.519970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a3aa2-a063-4ffb-9318-ee77066079a0 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:44.557872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 983/1969] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:44.559730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a3aa2-a063-4ffb-9318-ee77066079a0 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:44.592466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a3aa2-a063-4ffb-9318-ee77066079a0 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:44.602441 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed780917-8f27-40ae-bef6-40c313b33097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:44.603156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780917-8f27-40ae-bef6-40c313b33097 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.603439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780917-8f27-40ae-bef6-40c313b33097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.603720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed780917-8f27-40ae-bef6-40c313b33097 admin admin] PATH_INFO: `/v3/projects/3bfa756694494ed799b2d7a18f75edfd/users/71a07c10a77e4817b07701888a45b109/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.606103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780917-8f27-40ae-bef6-40c313b33097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bfa756694494ed799b2d7a18f75edfd, user_id=71a07c10a77e4817b07701888a45b109, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:44.609265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed780917-8f27-40ae-bef6-40c313b33097 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:44.643549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 984/1970] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:44 2022] PUT /identity/v3/projects/3bfa756694494ed799b2d7a18f75edfd/users/71a07c10a77e4817b07701888a45b109/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:44.645107 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77a3b7-e613-4e20-80e9-1eb620332def None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.646620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77a3b7-e613-4e20-80e9-1eb620332def None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.646922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77a3b7-e613-4e20-80e9-1eb620332def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.706396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 987/1971] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:05:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:44.715006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4f77a3b7-e613-4e20-80e9-1eb620332def None None] MFA Rules not processed for user `71a07c10a77e4817b07701888a45b109`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:44.715896 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4f77a3b7-e613-4e20-80e9-1eb620332def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:44.729294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14751194-608a-4478-95d8-cfdf4c14dbdb None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:44.730876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-14751194-608a-4478-95d8-cfdf4c14dbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:44.755420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14751194-608a-4478-95d8-cfdf4c14dbdb None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:44.785689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:44.815945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 985/1972] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:44.831577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77a3b7-e613-4e20-80e9-1eb620332def None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:44.833354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4f77a3b7-e613-4e20-80e9-1eb620332def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:44.910696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f77a3b7-e613-4e20-80e9-1eb620332def None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:44.917016 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709b9ba-6d3f-4d62-a752-d8606489a512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:44.918678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709b9ba-6d3f-4d62-a752-d8606489a512 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.920164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709b9ba-6d3f-4d62-a752-d8606489a512 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.920477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709b9ba-6d3f-4d62-a752-d8606489a512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.933793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba97dc8-19b4-41c5-a74f-f069a414b9bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:44.934331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709b9ba-6d3f-4d62-a752-d8606489a512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:44.934478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba97dc8-19b4-41c5-a74f-f069a414b9bf service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:44.934710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba97dc8-19b4-41c5-a74f-f069a414b9bf service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:44.934987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba97dc8-19b4-41c5-a74f-f069a414b9bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:44.935816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709b9ba-6d3f-4d62-a752-d8606489a512 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:44.946264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba97dc8-19b4-41c5-a74f-f069a414b9bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:44.947832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 986/1973] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:44.947958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba97dc8-19b4-41c5-a74f-f069a414b9bf service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:44.989214 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 988/1974] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:45.257177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709b9ba-6d3f-4d62-a752-d8606489a512 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:45.259962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4709b9ba-6d3f-4d62-a752-d8606489a512 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:45.450424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709b9ba-6d3f-4d62-a752-d8606489a512 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:45.471969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:45.473275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:45.473666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:45.474245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] PATH_INFO: `/v3/users/2af6962653584e16b9f5e35898142794` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:45.476102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af6962653584e16b9f5e35898142794)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:45.478322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:45.539239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af6962653584e16b9f5e35898142794'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:45.540015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af6962653584e16b9f5e35898142794'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:45.548271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af6962653584e16b9f5e35898142794'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:45.561320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af6962653584e16b9f5e35898142794'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:45.569837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af6962653584e16b9f5e35898142794'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:45.577366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 987/1975] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:45 2022] DELETE /identity/v3/users/2af6962653584e16b9f5e35898142794 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:45.582629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588863e-ccc2-4674-a92d-02d67a04dc70 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:45.583660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588863e-ccc2-4674-a92d-02d67a04dc70 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:45.584067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588863e-ccc2-4674-a92d-02d67a04dc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:45.609502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-0588863e-ccc2-4674-a92d-02d67a04dc70 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:45.611462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0588863e-ccc2-4674-a92d-02d67a04dc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:45.670994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:45.690420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 989/1976] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:45.715604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:45.718409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:45.733020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:45.762065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588863e-ccc2-4674-a92d-02d67a04dc70 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:45.764514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0588863e-ccc2-4674-a92d-02d67a04dc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:45.784849 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588863e-ccc2-4674-a92d-02d67a04dc70 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:45.796014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:45.796876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:45.797343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:45.797836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] PATH_INFO: `/v3/users/e72b440d371a40a589dc00b1ffaf2fda` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:45.804633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e72b440d371a40a589dc00b1ffaf2fda)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:45.807771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:45.824059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75be60-6ec9-40e3-87d2-4a413f208566 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:45.831070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ece46c-8fc6-4c40-b604-0f702bbd0d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:45.831824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ece46c-8fc6-4c40-b604-0f702bbd0d89 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:45.832072 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ece46c-8fc6-4c40-b604-0f702bbd0d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:45.832313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ece46c-8fc6-4c40-b604-0f702bbd0d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:45.873516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ece46c-8fc6-4c40-b604-0f702bbd0d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:45.874996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ece46c-8fc6-4c40-b604-0f702bbd0d89 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:45.909670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 988/1977] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:45.911156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e72b440d371a40a589dc00b1ffaf2fda'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:45.927862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e72b440d371a40a589dc00b1ffaf2fda'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:45.938627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e72b440d371a40a589dc00b1ffaf2fda'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:45.939633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e72b440d371a40a589dc00b1ffaf2fda'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:45.940507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e72b440d371a40a589dc00b1ffaf2fda'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:45.974077 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 990/1978] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:45 2022] DELETE /identity/v3/users/e72b440d371a40a589dc00b1ffaf2fda => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:45.978887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff3d1-111a-4a42-8d8b-427363c6eb5e None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:45.982426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff3d1-111a-4a42-8d8b-427363c6eb5e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:45.982426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff3d1-111a-4a42-8d8b-427363c6eb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:46.009955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9d0ff3d1-111a-4a42-8d8b-427363c6eb5e None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:46.011489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9d0ff3d1-111a-4a42-8d8b-427363c6eb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:46.130006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 989/1979] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:46.141858 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:46.143464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:46.249932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1aafe2-e5ad-4abb-9447-9df860a55ad8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:46.254401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca42ad2-489e-4cc5-9749-47ee4a32c603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:46.255208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42ad2-489e-4cc5-9749-47ee4a32c603 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:46.255553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42ad2-489e-4cc5-9749-47ee4a32c603 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:46.255894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca42ad2-489e-4cc5-9749-47ee4a32c603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:46.269280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca42ad2-489e-4cc5-9749-47ee4a32c603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:46.271278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca42ad2-489e-4cc5-9749-47ee4a32c603 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:46.303666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 991/1980] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:46.589644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ff3d1-111a-4a42-8d8b-427363c6eb5e None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:46.591257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9d0ff3d1-111a-4a42-8d8b-427363c6eb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:46.611584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ff3d1-111a-4a42-8d8b-427363c6eb5e None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:46.625026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:46.625699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:46.625992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:46.626245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] PATH_INFO: `/v3/projects/7c4d9bb518704c39a86f8c50265bbeb5` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:46.627469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c4d9bb518704c39a86f8c50265bbeb5)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:46.629549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:46.687325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c4d9bb518704c39a86f8c50265bbeb5', 'request_id': 'req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.695759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c4d9bb518704c39a86f8c50265bbeb5', 'request_id': 'req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.711761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c4d9bb518704c39a86f8c50265bbeb5', 'request_id': 'req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.727636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] The token cache is being invalidate because project 7c4d9bb518704c39a86f8c50265bbeb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:46.728360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.729488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.734892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 990/1981] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:46 2022] DELETE /identity/v3/projects/7c4d9bb518704c39a86f8c50265bbeb5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:46.735766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca42ad2-489e-4cc5-9749-47ee4a32c603 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:46.763524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca42ad2-489e-4cc5-9749-47ee4a32c603 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:46.875129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:46.881295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:46.884215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:46.884937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:46.885231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:46.885814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] PATH_INFO: `/v3/users/71a07c10a77e4817b07701888a45b109` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:46.887108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a07c10a77e4817b07701888a45b109)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:46.889058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:46.899815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d2693-b866-4e7c-9cd4-9cd5254aaf1f admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:46.904829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:46.905509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:46.905792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:46.906018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:46.906174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] PATH_INFO: `/v3/projects/1bbc1e3041b945319aa331522463318c` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:46.907329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bbc1e3041b945319aa331522463318c)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:46.909741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:46.946169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a07c10a77e4817b07701888a45b109'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.947808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:46.959968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a07c10a77e4817b07701888a45b109'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.965569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bbc1e3041b945319aa331522463318c', 'request_id': 'req-c932586b-92db-4cc3-8e01-f8fb0fb74fda'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.967093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a07c10a77e4817b07701888a45b109'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.967853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a07c10a77e4817b07701888a45b109'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.968585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a07c10a77e4817b07701888a45b109'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.971854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bbc1e3041b945319aa331522463318c', 'request_id': 'req-c932586b-92db-4cc3-8e01-f8fb0fb74fda'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.981185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bbc1e3041b945319aa331522463318c', 'request_id': 'req-c932586b-92db-4cc3-8e01-f8fb0fb74fda'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.984420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 992/1982] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:46 2022] DELETE /identity/v3/users/71a07c10a77e4817b07701888a45b109 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:46.992752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] The token cache is being invalidate because project 1bbc1e3041b945319aa331522463318c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:46.993044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.994080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:46.995655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 991/1983] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:46 2022] DELETE /identity/v3/projects/1bbc1e3041b945319aa331522463318c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:46.999768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:47.001260 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:47.080887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb15ad-e798-4a04-969f-9e68723fc533 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:47.087273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:47.088026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:47.088340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:47.089084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] PATH_INFO: `/v3/users/b8bb42fd2a2b45238e2d407540f6f295` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:47.090324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8bb42fd2a2b45238e2d407540f6f295)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:47.092315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:47.153923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8bb42fd2a2b45238e2d407540f6f295'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.166798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8bb42fd2a2b45238e2d407540f6f295'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.174320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8bb42fd2a2b45238e2d407540f6f295'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.175085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8bb42fd2a2b45238e2d407540f6f295'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.175699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8bb42fd2a2b45238e2d407540f6f295'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.190651 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 993/1984] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:46 2022] DELETE /identity/v3/users/b8bb42fd2a2b45238e2d407540f6f295 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:47.296974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:47.298578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:47.410366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932586b-92db-4cc3-8e01-f8fb0fb74fda admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:47.491174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:47.491865 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:47.492142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:47.493075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] PATH_INFO: `/v3/projects/3bfa756694494ed799b2d7a18f75edfd` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:47.495488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bfa756694494ed799b2d7a18f75edfd)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:47.498588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:47.557832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:47.559360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:47.560796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bfa756694494ed799b2d7a18f75edfd', 'request_id': 'req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.570292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bfa756694494ed799b2d7a18f75edfd', 'request_id': 'req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.573261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef54e7e-bc88-46cc-a95d-813aa1e21287 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:47.577600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:47.580178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bfa756694494ed799b2d7a18f75edfd', 'request_id': 'req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.585305 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:47.607272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] The token cache is being invalidate because project 3bfa756694494ed799b2d7a18f75edfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:47.607934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.608487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f0967e2e-853d-4cbd-9d85-11c90e8b7c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.610088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 992/1985] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:47 2022] DELETE /identity/v3/projects/3bfa756694494ed799b2d7a18f75edfd => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:47.616508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e582f-4603-4b17-b659-28f04bcc8652 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:47.616720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e582f-4603-4b17-b659-28f04bcc8652 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:47.617065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e582f-4603-4b17-b659-28f04bcc8652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:47.637601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6a5e582f-4603-4b17-b659-28f04bcc8652 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:47.638799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6a5e582f-4603-4b17-b659-28f04bcc8652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:47.648218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:47.648909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:47.649214 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:47.649489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] PATH_INFO: `/v3/projects/e5d9fb09f1a947e5a353bf5ed5b149cb` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:47.650759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d9fb09f1a947e5a353bf5ed5b149cb)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:47.652752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:47.698138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d9fb09f1a947e5a353bf5ed5b149cb', 'request_id': 'req-d8cb8d95-2b04-4eae-9ad3-6e6973491184'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.699882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 993/1986] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:47.704485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d9fb09f1a947e5a353bf5ed5b149cb', 'request_id': 'req-d8cb8d95-2b04-4eae-9ad3-6e6973491184'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.707877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc843671-9b33-4b0c-9b2b-28527a254714 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:47.708214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc843671-9b33-4b0c-9b2b-28527a254714 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:47.708670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc843671-9b33-4b0c-9b2b-28527a254714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:47.713769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d9fb09f1a947e5a353bf5ed5b149cb', 'request_id': 'req-d8cb8d95-2b04-4eae-9ad3-6e6973491184'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.724356 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] The token cache is being invalidate because project e5d9fb09f1a947e5a353bf5ed5b149cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:47.724651 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.725287 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d8cb8d95-2b04-4eae-9ad3-6e6973491184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:47.726819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 994/1987] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:47 2022] DELETE /identity/v3/projects/e5d9fb09f1a947e5a353bf5ed5b149cb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:47.731254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-fc843671-9b33-4b0c-9b2b-28527a254714 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:47.731770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304f316-0729-4fa9-9e18-339a0b858092 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:47.732055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304f316-0729-4fa9-9e18-339a0b858092 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:47.732359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304f316-0729-4fa9-9e18-339a0b858092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:47.732555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-fc843671-9b33-4b0c-9b2b-28527a254714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:47.757103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c304f316-0729-4fa9-9e18-339a0b858092 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:47.758369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c304f316-0729-4fa9-9e18-339a0b858092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:47.814566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 994/1988] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:47.833153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 995/1989] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:47.833362 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc843671-9b33-4b0c-9b2b-28527a254714 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:47.836331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efacbaf-8cf2-4236-a5a7-91d17d385dfa None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:47.836331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efacbaf-8cf2-4236-a5a7-91d17d385dfa None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:47.836331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efacbaf-8cf2-4236-a5a7-91d17d385dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:47.843221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc843671-9b33-4b0c-9b2b-28527a254714 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:47.852303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216db33-c824-42de-aa3f-807928ad7bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:47.853063 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216db33-c824-42de-aa3f-807928ad7bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:47.853229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216db33-c824-42de-aa3f-807928ad7bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:47.853507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e216db33-c824-42de-aa3f-807928ad7bfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:47.854227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216db33-c824-42de-aa3f-807928ad7bfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:47.854536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216db33-c824-42de-aa3f-807928ad7bfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:47.856183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e216db33-c824-42de-aa3f-807928ad7bfb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:47.871500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 995/1990] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:05:47.875503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216db33-c824-42de-aa3f-807928ad7bfb admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:47.875884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3efacbaf-8cf2-4236-a5a7-91d17d385dfa None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:47.878212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3efacbaf-8cf2-4236-a5a7-91d17d385dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:47.893825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e216db33-c824-42de-aa3f-807928ad7bfb admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:47.896088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 996/1991] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:47.904336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efacbaf-8cf2-4236-a5a7-91d17d385dfa None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:47.910225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fdf21-dd92-4ba3-a304-b1bdf4e477eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:47.910965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fdf21-dd92-4ba3-a304-b1bdf4e477eb admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:47.911400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fdf21-dd92-4ba3-a304-b1bdf4e477eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:47.911748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fdf21-dd92-4ba3-a304-b1bdf4e477eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:47.912532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fdf21-dd92-4ba3-a304-b1bdf4e477eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:47.915000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efacbaf-8cf2-4236-a5a7-91d17d385dfa None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:47.918190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998fdf21-dd92-4ba3-a304-b1bdf4e477eb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:47.920932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c57e8-6f5e-47c2-be06-1227fd5b53dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:47.921878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c57e8-6f5e-47c2-be06-1227fd5b53dc admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:47.922523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c57e8-6f5e-47c2-be06-1227fd5b53dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:47.922973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c57e8-6f5e-47c2-be06-1227fd5b53dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:47.923866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c57e8-6f5e-47c2-be06-1227fd5b53dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:47.924373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c57e8-6f5e-47c2-be06-1227fd5b53dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:47.926194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c57e8-6f5e-47c2-be06-1227fd5b53dc admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:47.952687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 997/1992] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:05:47.957165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c57e8-6f5e-47c2-be06-1227fd5b53dc admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:47.957311 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 996/1993] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:47 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:47.965073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fdf21-dd92-4ba3-a304-b1bdf4e477eb admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:47.969053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c57e8-6f5e-47c2-be06-1227fd5b53dc admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:47.985538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:48.046756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998fdf21-dd92-4ba3-a304-b1bdf4e477eb admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.050270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2b2e1-9d89-419e-9590-442ad3aa4d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.051325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2b2e1-9d89-419e-9590-442ad3aa4d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.051865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2b2e1-9d89-419e-9590-442ad3aa4d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.052019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bad835-7b02-43f4-81d5-2001f9c6e47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.052534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2b2e1-9d89-419e-9590-442ad3aa4d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.052694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad835-7b02-43f4-81d5-2001f9c6e47e admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.052973 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad835-7b02-43f4-81d5-2001f9c6e47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.053240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad835-7b02-43f4-81d5-2001f9c6e47e admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.053975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bad835-7b02-43f4-81d5-2001f9c6e47e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.054099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2b2e1-9d89-419e-9590-442ad3aa4d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.056074 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bad835-7b02-43f4-81d5-2001f9c6e47e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.058205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2b2e1-9d89-419e-9590-442ad3aa4d60 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.064979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:48.091189 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 997/1994] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:47 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:48.092375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bad835-7b02-43f4-81d5-2001f9c6e47e admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.105229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bad835-7b02-43f4-81d5-2001f9c6e47e admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.110385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42f9bf-d142-4c8e-aa8b-327387249a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.111131 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42f9bf-d142-4c8e-aa8b-327387249a3f admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.111520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42f9bf-d142-4c8e-aa8b-327387249a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.111873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42f9bf-d142-4c8e-aa8b-327387249a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.112613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42f9bf-d142-4c8e-aa8b-327387249a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.114557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42f9bf-d142-4c8e-aa8b-327387249a3f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.117784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 998/1995] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:47 2022] POST /identity/v3/projects => generated 409 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:48.122829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2b2e1-9d89-419e-9590-442ad3aa4d60 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.132185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 998/1996] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:48.133459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42f9bf-d142-4c8e-aa8b-327387249a3f admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.143089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42f9bf-d142-4c8e-aa8b-327387249a3f admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.195584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2b2e1-9d89-419e-9590-442ad3aa4d60 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.200056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c161f-64ca-4988-b8f1-d1d1b5c07d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.200429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd1eda-1436-4c52-82f1-164505c34e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.200844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c161f-64ca-4988-b8f1-d1d1b5c07d80 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.201063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd1eda-1436-4c52-82f1-164505c34e43 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.201177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c161f-64ca-4988-b8f1-d1d1b5c07d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.201326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd1eda-1436-4c52-82f1-164505c34e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.201416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c161f-64ca-4988-b8f1-d1d1b5c07d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.201606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd1eda-1436-4c52-82f1-164505c34e43 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.202062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c161f-64ca-4988-b8f1-d1d1b5c07d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.202282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd1eda-1436-4c52-82f1-164505c34e43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.203657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c161f-64ca-4988-b8f1-d1d1b5c07d80 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.204225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd1eda-1436-4c52-82f1-164505c34e43 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.217825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 999/1997] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:05:48.219020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c161f-64ca-4988-b8f1-d1d1b5c07d80 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.230379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 999/1998] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:48 2022] POST /identity/v3/users => generated 305 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:48.231378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c161f-64ca-4988-b8f1-d1d1b5c07d80 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.234512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd1eda-1436-4c52-82f1-164505c34e43 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.236418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33791125-eef8-423a-9d91-9fefddd74027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.237376 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33791125-eef8-423a-9d91-9fefddd74027 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.237376 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33791125-eef8-423a-9d91-9fefddd74027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.237645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33791125-eef8-423a-9d91-9fefddd74027 admin admin] PATH_INFO: `/v3/projects/cf11b64bfbae4f33a561c4af4d62b54c/users/a901329dac1a410a829928c0aa5f2f26/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.245693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd1eda-1436-4c52-82f1-164505c34e43 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.251158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7607d-dea3-4c81-b3ed-2d86fb7c5691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.251817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7607d-dea3-4c81-b3ed-2d86fb7c5691 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.252081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7607d-dea3-4c81-b3ed-2d86fb7c5691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.252392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7607d-dea3-4c81-b3ed-2d86fb7c5691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.253097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7607d-dea3-4c81-b3ed-2d86fb7c5691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.253733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33791125-eef8-423a-9d91-9fefddd74027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf11b64bfbae4f33a561c4af4d62b54c, user_id=a901329dac1a410a829928c0aa5f2f26, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.254728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7607d-dea3-4c81-b3ed-2d86fb7c5691 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.256493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33791125-eef8-423a-9d91-9fefddd74027 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.267306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1000/1999] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:05:48.271934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7607d-dea3-4c81-b3ed-2d86fb7c5691 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.278103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1000/2000] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:48 2022] PUT /identity/v3/projects/cf11b64bfbae4f33a561c4af4d62b54c/users/a901329dac1a410a829928c0aa5f2f26/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:48.288780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506451-a405-4068-9cd2-3788ef6a2831 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.289388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506451-a405-4068-9cd2-3788ef6a2831 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.289672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75506451-a405-4068-9cd2-3788ef6a2831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.291157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7607d-dea3-4c81-b3ed-2d86fb7c5691 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.322088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-75506451-a405-4068-9cd2-3788ef6a2831 None None] MFA Rules not processed for user `a901329dac1a410a829928c0aa5f2f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:48.323341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-75506451-a405-4068-9cd2-3788ef6a2831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:48.365522 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc795e50-a3c8-43e1-af02-03247d19361c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.366073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795e50-a3c8-43e1-af02-03247d19361c admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.366499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795e50-a3c8-43e1-af02-03247d19361c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.366781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795e50-a3c8-43e1-af02-03247d19361c admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.367490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc795e50-a3c8-43e1-af02-03247d19361c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.369690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc795e50-a3c8-43e1-af02-03247d19361c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.382005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1001/2001] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:48.385795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc795e50-a3c8-43e1-af02-03247d19361c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.403341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc795e50-a3c8-43e1-af02-03247d19361c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.410662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed900d4-8986-4d62-b764-d4f23844ee41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.411502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed900d4-8986-4d62-b764-d4f23844ee41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.411787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed900d4-8986-4d62-b764-d4f23844ee41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.412063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed900d4-8986-4d62-b764-d4f23844ee41 admin admin] PATH_INFO: `/v3/projects/d5b9828957b945c8a68b2cede2237450/users/3a14058bf2284c64b527e9c849c6b72b/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.428438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1001/2002] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:48.432946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75506451-a405-4068-9cd2-3788ef6a2831 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.434446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed900d4-8986-4d62-b764-d4f23844ee41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b9828957b945c8a68b2cede2237450, user_id=3a14058bf2284c64b527e9c849c6b72b, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.437132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed900d4-8986-4d62-b764-d4f23844ee41 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.440025 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:48.448214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75506451-a405-4068-9cd2-3788ef6a2831 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.464378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1002/2003] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:48 2022] PUT /identity/v3/projects/d5b9828957b945c8a68b2cede2237450/users/3a14058bf2284c64b527e9c849c6b72b/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:48.468328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b52a4-8627-48e8-b2b8-fbb186d09a0e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.468688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b52a4-8627-48e8-b2b8-fbb186d09a0e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.469032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b52a4-8627-48e8-b2b8-fbb186d09a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.496241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:48.501697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1c6b52a4-8627-48e8-b2b8-fbb186d09a0e None None] MFA Rules not processed for user `3a14058bf2284c64b527e9c849c6b72b`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:48.503749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1c6b52a4-8627-48e8-b2b8-fbb186d09a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:48.524125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154afd3a-f625-476c-bb13-32088a3628f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.524936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afd3a-f625-476c-bb13-32088a3628f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.525217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afd3a-f625-476c-bb13-32088a3628f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.525489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afd3a-f625-476c-bb13-32088a3628f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.526234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154afd3a-f625-476c-bb13-32088a3628f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.528406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154afd3a-f625-476c-bb13-32088a3628f0 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.570614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1002/2004] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:48 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:48.575388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154afd3a-f625-476c-bb13-32088a3628f0 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.605361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1003/2005] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:48 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:48.606735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b52a4-8627-48e8-b2b8-fbb186d09a0e None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.634506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b52a4-8627-48e8-b2b8-fbb186d09a0e None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.654745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154afd3a-f625-476c-bb13-32088a3628f0 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.661017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988b234-6bcc-4e69-a06f-df9b6823ba09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.661932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b234-6bcc-4e69-a06f-df9b6823ba09 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.662065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b234-6bcc-4e69-a06f-df9b6823ba09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.662271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988b234-6bcc-4e69-a06f-df9b6823ba09 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.663272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988b234-6bcc-4e69-a06f-df9b6823ba09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.665694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988b234-6bcc-4e69-a06f-df9b6823ba09 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.702427 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1003/2006] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:48 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:48.704776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988b234-6bcc-4e69-a06f-df9b6823ba09 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.710535 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40a318-e35f-4984-b931-e76f3374a6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.711586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40a318-e35f-4984-b931-e76f3374a6ea admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.712094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40a318-e35f-4984-b931-e76f3374a6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.713338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40a318-e35f-4984-b931-e76f3374a6ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.714163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40a318-e35f-4984-b931-e76f3374a6ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.717533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f40a318-e35f-4984-b931-e76f3374a6ea admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.723634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988b234-6bcc-4e69-a06f-df9b6823ba09 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.730807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911caf35-7838-4f80-8153-2b11e049ebf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.731566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911caf35-7838-4f80-8153-2b11e049ebf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.731848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911caf35-7838-4f80-8153-2b11e049ebf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.732120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911caf35-7838-4f80-8153-2b11e049ebf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.733033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911caf35-7838-4f80-8153-2b11e049ebf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.735893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911caf35-7838-4f80-8153-2b11e049ebf4 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.753521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1004/2007] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:48.754619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911caf35-7838-4f80-8153-2b11e049ebf4 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.762709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1004/2008] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:48 2022] POST /identity/v3/projects => generated 409 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:48.764234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40a318-e35f-4984-b931-e76f3374a6ea admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.775186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911caf35-7838-4f80-8153-2b11e049ebf4 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.856454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002438f-6a96-4931-94fe-4cf82d218d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.857678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002438f-6a96-4931-94fe-4cf82d218d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.857678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002438f-6a96-4931-94fe-4cf82d218d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.857990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002438f-6a96-4931-94fe-4cf82d218d10 admin admin] PATH_INFO: `/v3/projects/6851b56a3e2c43bfacef6de5ef66c507/users/984468c1164047d4bfff9330b6084a04/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.877290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40a318-e35f-4984-b931-e76f3374a6ea admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.884343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a92ea5-90c3-465d-8ce6-ec021e739059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:48.884591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002438f-6a96-4931-94fe-4cf82d218d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6851b56a3e2c43bfacef6de5ef66c507, user_id=984468c1164047d4bfff9330b6084a04, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.885939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a92ea5-90c3-465d-8ce6-ec021e739059 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:48.886403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a92ea5-90c3-465d-8ce6-ec021e739059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:48.886815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a92ea5-90c3-465d-8ce6-ec021e739059 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:48.887797 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a92ea5-90c3-465d-8ce6-ec021e739059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:48.891137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a92ea5-90c3-465d-8ce6-ec021e739059 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.906706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002438f-6a96-4931-94fe-4cf82d218d10 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:48.911983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:48.924372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1005/2009] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:48 2022] PUT /identity/v3/projects/6851b56a3e2c43bfacef6de5ef66c507/users/984468c1164047d4bfff9330b6084a04/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:48.929498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002438f-6a96-4931-94fe-4cf82d218d10 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.933202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1005/2010] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:48 2022] POST /identity/v3/users => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:48.941379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a92ea5-90c3-465d-8ce6-ec021e739059 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:48.955256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a92ea5-90c3-465d-8ce6-ec021e739059 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:48.969768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:49.015134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002438f-6a96-4931-94fe-4cf82d218d10 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:49.021070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe27fc1-053c-4a54-b87c-6017effd67c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:49.021540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe27fc1-053c-4a54-b87c-6017effd67c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.021883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe27fc1-053c-4a54-b87c-6017effd67c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.022156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe27fc1-053c-4a54-b87c-6017effd67c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.022864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe27fc1-053c-4a54-b87c-6017effd67c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:49.024535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe27fc1-053c-4a54-b87c-6017effd67c5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:49.040094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1006/2011] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:49.043568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe27fc1-053c-4a54-b87c-6017effd67c5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:49.047315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9487f-0540-4aa9-88ba-d309ad027691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:49.047979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9487f-0540-4aa9-88ba-d309ad027691 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.048318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9487f-0540-4aa9-88ba-d309ad027691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.048553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9487f-0540-4aa9-88ba-d309ad027691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.049419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9487f-0540-4aa9-88ba-d309ad027691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:49.051451 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9487f-0540-4aa9-88ba-d309ad027691 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:49.055479 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe27fc1-053c-4a54-b87c-6017effd67c5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:49.062570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de989955-ca10-4eb4-b565-f4bf5470766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:49.063872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989955-ca10-4eb4-b565-f4bf5470766e admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.063872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989955-ca10-4eb4-b565-f4bf5470766e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.064084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de989955-ca10-4eb4-b565-f4bf5470766e admin admin] PATH_INFO: `/v3/projects/6851b56a3e2c43bfacef6de5ef66c507/users/984468c1164047d4bfff9330b6084a04/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.071901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1006/2012] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:05:49.072088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de989955-ca10-4eb4-b565-f4bf5470766e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6851b56a3e2c43bfacef6de5ef66c507, user_id=984468c1164047d4bfff9330b6084a04, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:49.073309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9487f-0540-4aa9-88ba-d309ad027691 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:49.080257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de989955-ca10-4eb4-b565-f4bf5470766e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:49.085017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9487f-0540-4aa9-88ba-d309ad027691 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:49.090186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953ecf5-f776-4a41-9cd2-0b9ce27448a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:49.091263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953ecf5-f776-4a41-9cd2-0b9ce27448a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.091710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953ecf5-f776-4a41-9cd2-0b9ce27448a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.092133 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f953ecf5-f776-4a41-9cd2-0b9ce27448a5 admin admin] PATH_INFO: `/v3/projects/5a4001a57db14c1f81db972a1b72e1bd/users/8d1dfc7e85844e9eb8ccbb2c5b5063b8/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.101491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1007/2013] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:49 2022] PUT /identity/v3/projects/6851b56a3e2c43bfacef6de5ef66c507/users/984468c1164047d4bfff9330b6084a04/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:49.112881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c60212-f2f5-46da-a310-9e4cc63483b9 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.119564 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953ecf5-f776-4a41-9cd2-0b9ce27448a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4001a57db14c1f81db972a1b72e1bd, user_id=8d1dfc7e85844e9eb8ccbb2c5b5063b8, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:49.121004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c60212-f2f5-46da-a310-9e4cc63483b9 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.121397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c60212-f2f5-46da-a310-9e4cc63483b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.123822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f953ecf5-f776-4a41-9cd2-0b9ce27448a5 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:49.145030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1007/2014] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:49 2022] PUT /identity/v3/projects/5a4001a57db14c1f81db972a1b72e1bd/users/8d1dfc7e85844e9eb8ccbb2c5b5063b8/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:49.149600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953ecf5-f776-4a41-9cd2-0b9ce27448a5 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:49.160731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-98c60212-f2f5-46da-a310-9e4cc63483b9 None None] MFA Rules not processed for user `984468c1164047d4bfff9330b6084a04`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:49.162177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-98c60212-f2f5-46da-a310-9e4cc63483b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:49.253845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f953ecf5-f776-4a41-9cd2-0b9ce27448a5 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:49.256245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1008/2015] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:49.259013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff8afd-49b4-4286-aaa5-3fc223490904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:49.259650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff8afd-49b4-4286-aaa5-3fc223490904 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.259998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff8afd-49b4-4286-aaa5-3fc223490904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.260269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff8afd-49b4-4286-aaa5-3fc223490904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.261016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff8afd-49b4-4286-aaa5-3fc223490904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:49.262636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff8afd-49b4-4286-aaa5-3fc223490904 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:49.267832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c60212-f2f5-46da-a310-9e4cc63483b9 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:49.269360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-98c60212-f2f5-46da-a310-9e4cc63483b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:49.277804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1008/2016] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:49.279069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff8afd-49b4-4286-aaa5-3fc223490904 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:49.289699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff8afd-49b4-4286-aaa5-3fc223490904 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:49.294624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e19482-83fc-4f69-8882-32deb839d583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:49.295311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e19482-83fc-4f69-8882-32deb839d583 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.295591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e19482-83fc-4f69-8882-32deb839d583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.295868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e19482-83fc-4f69-8882-32deb839d583 admin admin] PATH_INFO: `/v3/projects/5a4001a57db14c1f81db972a1b72e1bd/users/8d1dfc7e85844e9eb8ccbb2c5b5063b8/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.297917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e19482-83fc-4f69-8882-32deb839d583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4001a57db14c1f81db972a1b72e1bd, user_id=8d1dfc7e85844e9eb8ccbb2c5b5063b8, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:49.300668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e19482-83fc-4f69-8882-32deb839d583 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:49.314207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1009/2017] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:49 2022] PUT /identity/v3/projects/5a4001a57db14c1f81db972a1b72e1bd/users/8d1dfc7e85844e9eb8ccbb2c5b5063b8/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:49.326274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8b670-45cc-4027-96e8-049777f4da02 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.326551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8b670-45cc-4027-96e8-049777f4da02 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.326937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8b670-45cc-4027-96e8-049777f4da02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.344746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c60212-f2f5-46da-a310-9e4cc63483b9 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:49.361058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b9e8b670-45cc-4027-96e8-049777f4da02 None None] MFA Rules not processed for user `8d1dfc7e85844e9eb8ccbb2c5b5063b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:49.362794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b9e8b670-45cc-4027-96e8-049777f4da02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:49.405863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aafd33-b164-4120-8ed5-199c2f61a450 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:49.407447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aafd33-b164-4120-8ed5-199c2f61a450 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.407748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aafd33-b164-4120-8ed5-199c2f61a450 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.408028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aafd33-b164-4120-8ed5-199c2f61a450 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.410965 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:49.415435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:49.444785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1010/2018] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:49 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:49.460122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8b670-45cc-4027-96e8-049777f4da02 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:49.461890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b9e8b670-45cc-4027-96e8-049777f4da02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:49.476721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8b670-45cc-4027-96e8-049777f4da02 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:49.481020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bf767-7d76-451c-b218-efae60582c48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:49.481701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bf767-7d76-451c-b218-efae60582c48 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.482120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bf767-7d76-451c-b218-efae60582c48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.482396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bf767-7d76-451c-b218-efae60582c48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.492778 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38bf767-7d76-451c-b218-efae60582c48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:49.494613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38bf767-7d76-451c-b218-efae60582c48 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:49.497495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aafd33-b164-4120-8ed5-199c2f61a450 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:49.499787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aafd33-b164-4120-8ed5-199c2f61a450 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:49.505863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1011/2019] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:49 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:05:49.512978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1009/2020] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:05:49.665058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bf767-7d76-451c-b218-efae60582c48 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:49.675776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bf767-7d76-451c-b218-efae60582c48 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:49.683164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaddb01-d580-4495-bd1b-f08fe5e8a7c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:49.684238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaddb01-d580-4495-bd1b-f08fe5e8a7c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:49.684721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaddb01-d580-4495-bd1b-f08fe5e8a7c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:49.685190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaddb01-d580-4495-bd1b-f08fe5e8a7c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:49.770617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaddb01-d580-4495-bd1b-f08fe5e8a7c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:49.772463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaddb01-d580-4495-bd1b-f08fe5e8a7c6 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:49.785601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1012/2021] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:49 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:50.277241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aafd33-b164-4120-8ed5-199c2f61a450 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:50.289806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aafd33-b164-4120-8ed5-199c2f61a450 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:50.294516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee401225-2862-4ae2-8154-35afb64770d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:50.296071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee401225-2862-4ae2-8154-35afb64770d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:50.296407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee401225-2862-4ae2-8154-35afb64770d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:50.296689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee401225-2862-4ae2-8154-35afb64770d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:50.298657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ee401225-2862-4ae2-8154-35afb64770d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:50.396528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee401225-2862-4ae2-8154-35afb64770d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:50.398121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee401225-2862-4ae2-8154-35afb64770d4 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:50.407924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1010/2022] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:52.211834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaddb01-d580-4495-bd1b-f08fe5e8a7c6 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:52.286648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaddb01-d580-4495-bd1b-f08fe5e8a7c6 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:52.291709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:52.292419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:52.292703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:52.292974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] PATH_INFO: `/v3/users/a901329dac1a410a829928c0aa5f2f26` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:52.294238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a901329dac1a410a829928c0aa5f2f26)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:52.296247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:52.360025 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a901329dac1a410a829928c0aa5f2f26'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:52.374059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a901329dac1a410a829928c0aa5f2f26'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:52.375584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:52.387022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a901329dac1a410a829928c0aa5f2f26'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:52.387785 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a901329dac1a410a829928c0aa5f2f26'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:52.388616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a901329dac1a410a829928c0aa5f2f26'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:52.405151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1013/2023] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:52 2022] DELETE /identity/v3/users/a901329dac1a410a829928c0aa5f2f26 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:52.409879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0624b-2110-4b89-a756-64d71b56be9a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:52.410380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0624b-2110-4b89-a756-64d71b56be9a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:52.410481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0624b-2110-4b89-a756-64d71b56be9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:52.433988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d3a0624b-2110-4b89-a756-64d71b56be9a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:52.435860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d3a0624b-2110-4b89-a756-64d71b56be9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:52.500487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1011/2024] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:52.510962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:52.512995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:52.581053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059f918-2eda-4eb0-a7d4-a137e398bfbd admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:52.585498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a726608-f6a2-4698-8da9-b16ea46f8253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:52.586143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a726608-f6a2-4698-8da9-b16ea46f8253 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:52.586418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a726608-f6a2-4698-8da9-b16ea46f8253 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:52.586690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a726608-f6a2-4698-8da9-b16ea46f8253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:52.600066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a726608-f6a2-4698-8da9-b16ea46f8253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:52.601621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a726608-f6a2-4698-8da9-b16ea46f8253 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:52.612210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1014/2025] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:52.829993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0624b-2110-4b89-a756-64d71b56be9a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:52.835152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d3a0624b-2110-4b89-a756-64d71b56be9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:52.847836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0624b-2110-4b89-a756-64d71b56be9a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:52.852803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:52.853998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:52.853998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:52.853998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] PATH_INFO: `/v3/users/01edd8cb1da94a39bcb162e0e292c41e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:52.855166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01edd8cb1da94a39bcb162e0e292c41e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:52.858299 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:52.905187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:52.913095 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01edd8cb1da94a39bcb162e0e292c41e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:52.916494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01edd8cb1da94a39bcb162e0e292c41e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:52.925507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01edd8cb1da94a39bcb162e0e292c41e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:52.938953 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01edd8cb1da94a39bcb162e0e292c41e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:52.950423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01edd8cb1da94a39bcb162e0e292c41e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:52.957129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1012/2026] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:52 2022] DELETE /identity/v3/users/01edd8cb1da94a39bcb162e0e292c41e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:52.962170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d183f-ea73-483d-831c-f41cb4cc52a6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:52.962637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d183f-ea73-483d-831c-f41cb4cc52a6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:52.963081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d183f-ea73-483d-831c-f41cb4cc52a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:52.985261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a56d183f-ea73-483d-831c-f41cb4cc52a6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:52.986505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a56d183f-ea73-483d-831c-f41cb4cc52a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:53.052066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1015/2027] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:53.064649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:53.066422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:53.143509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d183f-ea73-483d-831c-f41cb4cc52a6 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:53.145073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a56d183f-ea73-483d-831c-f41cb4cc52a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:53.158858 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d183f-ea73-483d-831c-f41cb4cc52a6 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:53.167498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:53.169752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:53.170235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:53.170721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] PATH_INFO: `/v3/projects/cf11b64bfbae4f33a561c4af4d62b54c` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:53.172988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf11b64bfbae4f33a561c4af4d62b54c)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:53.181091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:53.195806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d79a4-96ba-4053-b476-cb9d70e62711 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:53.203405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6aecc-e464-4909-ac1a-ae64aefbd5e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:53.204161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6aecc-e464-4909-ac1a-ae64aefbd5e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:53.204483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6aecc-e464-4909-ac1a-ae64aefbd5e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:53.204758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6aecc-e464-4909-ac1a-ae64aefbd5e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:53.216000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6aecc-e464-4909-ac1a-ae64aefbd5e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:53.219705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6aecc-e464-4909-ac1a-ae64aefbd5e7 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:53.233992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1013/2028] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:53.256150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf11b64bfbae4f33a561c4af4d62b54c', 'request_id': 'req-ec13312a-8b36-4ee2-b91c-e27d7a721629'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.266231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf11b64bfbae4f33a561c4af4d62b54c', 'request_id': 'req-ec13312a-8b36-4ee2-b91c-e27d7a721629'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.278503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf11b64bfbae4f33a561c4af4d62b54c', 'request_id': 'req-ec13312a-8b36-4ee2-b91c-e27d7a721629'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.305004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] The token cache is being invalidate because project cf11b64bfbae4f33a561c4af4d62b54c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:53.305310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.305984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.307645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1016/2029] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:53 2022] DELETE /identity/v3/projects/cf11b64bfbae4f33a561c4af4d62b54c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:53.312159 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6aecc-e464-4909-ac1a-ae64aefbd5e7 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:53.326416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6aecc-e464-4909-ac1a-ae64aefbd5e7 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:53.412015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:53.412729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:53.412983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:53.413257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] PATH_INFO: `/v3/users/984468c1164047d4bfff9330b6084a04` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:53.414515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=984468c1164047d4bfff9330b6084a04)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:53.416600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:53.477001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '984468c1164047d4bfff9330b6084a04'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.477820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '984468c1164047d4bfff9330b6084a04'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.486781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '984468c1164047d4bfff9330b6084a04'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.492541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:53.509777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '984468c1164047d4bfff9330b6084a04'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.518460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '984468c1164047d4bfff9330b6084a04'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.526893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1014/2030] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:53 2022] DELETE /identity/v3/users/984468c1164047d4bfff9330b6084a04 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:53.595475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:53.597121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:53.611767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec13312a-8b36-4ee2-b91c-e27d7a721629 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:53.620888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:53.723099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:53.725336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:53.725772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:53.726184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] PATH_INFO: `/v3/projects/8561ccadb0e94f528108c36bfe5f1415` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:53.731035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8561ccadb0e94f528108c36bfe5f1415)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:53.733148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:53.785354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8561ccadb0e94f528108c36bfe5f1415', 'request_id': 'req-2b8eace6-0d62-4566-933a-90b1f0cd8906'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.792076 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8561ccadb0e94f528108c36bfe5f1415', 'request_id': 'req-2b8eace6-0d62-4566-933a-90b1f0cd8906'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.800802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8561ccadb0e94f528108c36bfe5f1415', 'request_id': 'req-2b8eace6-0d62-4566-933a-90b1f0cd8906'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.812127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] The token cache is being invalidate because project 8561ccadb0e94f528108c36bfe5f1415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:53.812482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.813155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.814899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1017/2031] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:53 2022] DELETE /identity/v3/projects/8561ccadb0e94f528108c36bfe5f1415 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:53.818811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:53.820258 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:53.833683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103598a1-94dc-45c2-b9ed-fb1f8ea35514 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:53.878349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:53.880199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:53.909835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:53.910450 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:53.910704 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:53.910979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] PATH_INFO: `/v3/users/c35f0efeb0774e6fb5eebb7ca29ac83d` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:53.913610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c35f0efeb0774e6fb5eebb7ca29ac83d)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:53.917053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:53.958195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8eace6-0d62-4566-933a-90b1f0cd8906 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:53.963867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:53.964625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:53.965001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:53.965325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] PATH_INFO: `/v3/projects/6851b56a3e2c43bfacef6de5ef66c507` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:53.966875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6851b56a3e2c43bfacef6de5ef66c507)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:53.969056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:53.978849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c35f0efeb0774e6fb5eebb7ca29ac83d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.979624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c35f0efeb0774e6fb5eebb7ca29ac83d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:53.991226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c35f0efeb0774e6fb5eebb7ca29ac83d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.007996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c35f0efeb0774e6fb5eebb7ca29ac83d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.017797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1160489a-3227-4ad9-a9e4-0d9108326ad3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c35f0efeb0774e6fb5eebb7ca29ac83d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.020654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:54.029054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1015/2032] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:53 2022] DELETE /identity/v3/users/c35f0efeb0774e6fb5eebb7ca29ac83d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:54.038412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6851b56a3e2c43bfacef6de5ef66c507', 'request_id': 'req-e50e5523-5973-4f23-9fa4-50d31b819718'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.047130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6851b56a3e2c43bfacef6de5ef66c507', 'request_id': 'req-e50e5523-5973-4f23-9fa4-50d31b819718'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.056909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6851b56a3e2c43bfacef6de5ef66c507', 'request_id': 'req-e50e5523-5973-4f23-9fa4-50d31b819718'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.068687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] The token cache is being invalidate because project 6851b56a3e2c43bfacef6de5ef66c507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:54.068969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.069678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e50e5523-5973-4f23-9fa4-50d31b819718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.071288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1018/2033] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:53 2022] DELETE /identity/v3/projects/6851b56a3e2c43bfacef6de5ef66c507 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:54.077141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c65ccb-0975-4d56-b802-f47afcaf1865 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:54.077505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c65ccb-0975-4d56-b802-f47afcaf1865 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:54.077959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c65ccb-0975-4d56-b802-f47afcaf1865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:54.104349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d5c65ccb-0975-4d56-b802-f47afcaf1865 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:54.105842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d5c65ccb-0975-4d56-b802-f47afcaf1865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:54.190855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:54.206495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1016/2034] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:54.215722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07ee0e-f85a-4512-9c4a-c0bfe3755e0f None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:54.216372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07ee0e-f85a-4512-9c4a-c0bfe3755e0f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:54.216831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07ee0e-f85a-4512-9c4a-c0bfe3755e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:54.249570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-db07ee0e-f85a-4512-9c4a-c0bfe3755e0f None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:54.250853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-db07ee0e-f85a-4512-9c4a-c0bfe3755e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:54.259484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1019/2035] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:54.264040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c65ccb-0975-4d56-b802-f47afcaf1865 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:54.275499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c65ccb-0975-4d56-b802-f47afcaf1865 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:54.280389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a563a3-60b9-415b-8b71-2d0021cfd573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:54.281043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a563a3-60b9-415b-8b71-2d0021cfd573 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:54.281321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a563a3-60b9-415b-8b71-2d0021cfd573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:54.281594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a563a3-60b9-415b-8b71-2d0021cfd573 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:54.282424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a563a3-60b9-415b-8b71-2d0021cfd573 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:54.282730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a563a3-60b9-415b-8b71-2d0021cfd573 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:54.284467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a563a3-60b9-415b-8b71-2d0021cfd573 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:54.298043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1017/2036] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:05:54.304438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07ee0e-f85a-4512-9c4a-c0bfe3755e0f None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:54.321420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07ee0e-f85a-4512-9c4a-c0bfe3755e0f None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:54.328057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95bc45-d748-4d50-98be-f9e823e591e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:54.329064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95bc45-d748-4d50-98be-f9e823e591e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:54.329631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95bc45-d748-4d50-98be-f9e823e591e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:54.329904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95bc45-d748-4d50-98be-f9e823e591e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:54.330641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95bc45-d748-4d50-98be-f9e823e591e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:54.332645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95bc45-d748-4d50-98be-f9e823e591e1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:54.373182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1020/2037] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:54 2022] POST /identity/v3/projects => generated 391 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:54.381118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a563a3-60b9-415b-8b71-2d0021cfd573 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:54.400546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a563a3-60b9-415b-8b71-2d0021cfd573 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:54.473266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95bc45-d748-4d50-98be-f9e823e591e1 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:54.477062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-da95bc45-d748-4d50-98be-f9e823e591e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:54.528866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ef7aa-ae8f-4835-9c6d-cf50e8e73914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:54.529701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ef7aa-ae8f-4835-9c6d-cf50e8e73914 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:54.530097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ef7aa-ae8f-4835-9c6d-cf50e8e73914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:54.530435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ef7aa-ae8f-4835-9c6d-cf50e8e73914 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:54.532492 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ef7aa-ae8f-4835-9c6d-cf50e8e73914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:54.534994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ef7aa-ae8f-4835-9c6d-cf50e8e73914 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:54.565474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1018/2038] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:54 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:54.571181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ef7aa-ae8f-4835-9c6d-cf50e8e73914 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:54.584014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ef7aa-ae8f-4835-9c6d-cf50e8e73914 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:54.597796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c072e-3f0f-4500-8040-3933aa80f661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:54.598727 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c072e-3f0f-4500-8040-3933aa80f661 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:54.599238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c072e-3f0f-4500-8040-3933aa80f661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:54.599582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c072e-3f0f-4500-8040-3933aa80f661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:54.600554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c072e-3f0f-4500-8040-3933aa80f661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:54.603763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c072e-3f0f-4500-8040-3933aa80f661 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:54.614074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95bc45-d748-4d50-98be-f9e823e591e1 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:54.620124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1019/2039] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:05:54.623890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:54.625116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c072e-3f0f-4500-8040-3933aa80f661 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:54.625418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:54.625864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:54.626296 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] PATH_INFO: `/v3/projects/5afb7433d0f24444bb69fdd50f56c0c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:54.628539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afb7433d0f24444bb69fdd50f56c0c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:54.632989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:54.637564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c072e-3f0f-4500-8040-3933aa80f661 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:54.642753 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48d26f-353d-4b40-b328-95281a29162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:54.643550 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48d26f-353d-4b40-b328-95281a29162c admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:54.643934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48d26f-353d-4b40-b328-95281a29162c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:54.644324 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48d26f-353d-4b40-b328-95281a29162c admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:54.645157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48d26f-353d-4b40-b328-95281a29162c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:54.647511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48d26f-353d-4b40-b328-95281a29162c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:54.658238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1020/2040] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:54.663081 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48d26f-353d-4b40-b328-95281a29162c admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:54.697208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48d26f-353d-4b40-b328-95281a29162c admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:54.706548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4569d-2b07-4ede-a829-6eb215dab6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:54.707733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4569d-2b07-4ede-a829-6eb215dab6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:54.708236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4569d-2b07-4ede-a829-6eb215dab6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:54.708909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4569d-2b07-4ede-a829-6eb215dab6a5 admin admin] PATH_INFO: `/v3/projects/aec00f289f7b40a0985bfb73f0aab63a/users/b93fd3b624134a41ba7e2ad7564c363e/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:54.744908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afb7433d0f24444bb69fdd50f56c0c3', 'request_id': 'req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.756139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afb7433d0f24444bb69fdd50f56c0c3', 'request_id': 'req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.760916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4569d-2b07-4ede-a829-6eb215dab6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec00f289f7b40a0985bfb73f0aab63a, user_id=b93fd3b624134a41ba7e2ad7564c363e, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:54.768905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4569d-2b07-4ede-a829-6eb215dab6a5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:54.773226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afb7433d0f24444bb69fdd50f56c0c3', 'request_id': 'req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.796774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] The token cache is being invalidate because project 5afb7433d0f24444bb69fdd50f56c0c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:54.798186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.800558 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:54.802959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1021/2042] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:54 2022] DELETE /identity/v3/projects/5afb7433d0f24444bb69fdd50f56c0c3 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:54.804394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1021/2042] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:54 2022] PUT /identity/v3/projects/aec00f289f7b40a0985bfb73f0aab63a/users/b93fd3b624134a41ba7e2ad7564c363e/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:54.807591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:54.809432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:54.813331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da364ec-fa7d-4afe-bf06-f9edebace14c None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:54.813612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da364ec-fa7d-4afe-bf06-f9edebace14c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:54.813887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da364ec-fa7d-4afe-bf06-f9edebace14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:54.876636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7da364ec-fa7d-4afe-bf06-f9edebace14c None None] MFA Rules not processed for user `b93fd3b624134a41ba7e2ad7564c363e`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:54.878025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7da364ec-fa7d-4afe-bf06-f9edebace14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:54.892560 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:54.939894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7438965a-86fe-4b6d-aee1-0ac8ac9e6803 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:54.983815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1022/2043] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:54.995826 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da364ec-fa7d-4afe-bf06-f9edebace14c None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:54.997269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7da364ec-fa7d-4afe-bf06-f9edebace14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:55.022591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:55.046449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:55.048321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:55.048321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:55.048321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] PATH_INFO: `/v3/users/3fc1ad0f5c8643488da7eb9b6e90b431` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:55.050348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fc1ad0f5c8643488da7eb9b6e90b431)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:55.053960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:55.081146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da364ec-fa7d-4afe-bf06-f9edebace14c None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:55.088243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4d488-c2e5-4a2f-b828-fc32900ac526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:55.089153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4d488-c2e5-4a2f-b828-fc32900ac526 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:55.089606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4d488-c2e5-4a2f-b828-fc32900ac526 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:55.090018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4d488-c2e5-4a2f-b828-fc32900ac526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:55.101818 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4d488-c2e5-4a2f-b828-fc32900ac526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:55.104170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4d488-c2e5-4a2f-b828-fc32900ac526 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:55.118949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1023/2044] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:05:55.125160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fc1ad0f5c8643488da7eb9b6e90b431'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:55.140127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fc1ad0f5c8643488da7eb9b6e90b431'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:55.150341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fc1ad0f5c8643488da7eb9b6e90b431'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:55.151068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fc1ad0f5c8643488da7eb9b6e90b431'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:55.151718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3d60c218-3683-4f45-b311-09c5f3cee076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fc1ad0f5c8643488da7eb9b6e90b431'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:55.167974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1022/2045] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:54 2022] DELETE /identity/v3/users/3fc1ad0f5c8643488da7eb9b6e90b431 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:05:55.600823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4d488-c2e5-4a2f-b828-fc32900ac526 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:55.612594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4d488-c2e5-4a2f-b828-fc32900ac526 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:55.680098 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:55.681155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:55.681488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:55.681951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] PATH_INFO: `/v3/projects/93845215c0e541cc8180df404bdbada6` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:55.684419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=93845215c0e541cc8180df404bdbada6)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:55.686876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:55.749234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93845215c0e541cc8180df404bdbada6', 'request_id': 'req-688b53c2-f186-40cf-ac17-daff2f47d67a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:55.756054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93845215c0e541cc8180df404bdbada6', 'request_id': 'req-688b53c2-f186-40cf-ac17-daff2f47d67a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:55.777585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93845215c0e541cc8180df404bdbada6', 'request_id': 'req-688b53c2-f186-40cf-ac17-daff2f47d67a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:55.798255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] The token cache is being invalidate because project 93845215c0e541cc8180df404bdbada6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:05:55.798255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:55.798255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-688b53c2-f186-40cf-ac17-daff2f47d67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:05:55.798255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1024/2046] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:05:55 2022] DELETE /identity/v3/projects/93845215c0e541cc8180df404bdbada6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:55.799447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87c803-3053-4a39-89c5-185f669777a8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:55.799962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87c803-3053-4a39-89c5-185f669777a8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:55.800041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87c803-3053-4a39-89c5-185f669777a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:55.822489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-da87c803-3053-4a39-89c5-185f669777a8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:55.823738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-da87c803-3053-4a39-89c5-185f669777a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:55.886597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:55.912701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf99e-f0c1-401e-95e5-55139d1c31c2 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:55.913355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf99e-f0c1-401e-95e5-55139d1c31c2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:55.913649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cf99e-f0c1-401e-95e5-55139d1c31c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:55.928707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1023/2047] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:55.963657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8d7cf99e-f0c1-401e-95e5-55139d1c31c2 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:55.965935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8d7cf99e-f0c1-401e-95e5-55139d1c31c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:55.990219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1025/2048] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:05:55.994523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87c803-3053-4a39-89c5-185f669777a8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.007578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87c803-3053-4a39-89c5-185f669777a8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.015420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01799d2-afba-4163-830b-013a64c9682f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.016707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01799d2-afba-4163-830b-013a64c9682f admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.017221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01799d2-afba-4163-830b-013a64c9682f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.017723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01799d2-afba-4163-830b-013a64c9682f admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.019403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01799d2-afba-4163-830b-013a64c9682f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:05:56.020047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01799d2-afba-4163-830b-013a64c9682f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.023710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01799d2-afba-4163-830b-013a64c9682f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:56.048111 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1024/2049] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:05:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:05:56.052488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cf99e-f0c1-401e-95e5-55139d1c31c2 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.067907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cf99e-f0c1-401e-95e5-55139d1c31c2 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.074657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722a073-9a1a-497d-8163-1116dccd095e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.075624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722a073-9a1a-497d-8163-1116dccd095e admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.076086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722a073-9a1a-497d-8163-1116dccd095e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.076570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b722a073-9a1a-497d-8163-1116dccd095e admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.077455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722a073-9a1a-497d-8163-1116dccd095e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.080951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b722a073-9a1a-497d-8163-1116dccd095e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:56.143332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1026/2050] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:56 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:56.145172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01799d2-afba-4163-830b-013a64c9682f admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.159451 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01799d2-afba-4163-830b-013a64c9682f admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.297145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69b3b-c059-4092-8abb-d6b2d4de3216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.298139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69b3b-c059-4092-8abb-d6b2d4de3216 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.298519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69b3b-c059-4092-8abb-d6b2d4de3216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.298899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab69b3b-c059-4092-8abb-d6b2d4de3216 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.299847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab69b3b-c059-4092-8abb-d6b2d4de3216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.302953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab69b3b-c059-4092-8abb-d6b2d4de3216 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:56.333174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1025/2051] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:56 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:56.348835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722a073-9a1a-497d-8163-1116dccd095e admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.361759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b722a073-9a1a-497d-8163-1116dccd095e admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.366560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e640f5-a8e4-4967-8ed8-a56c72b96c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.367216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e640f5-a8e4-4967-8ed8-a56c72b96c0d admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.367637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e640f5-a8e4-4967-8ed8-a56c72b96c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.368046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e640f5-a8e4-4967-8ed8-a56c72b96c0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.369665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e640f5-a8e4-4967-8ed8-a56c72b96c0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.371311 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e640f5-a8e4-4967-8ed8-a56c72b96c0d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:56.383265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1027/2052] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:56.389490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69b3b-c059-4092-8abb-d6b2d4de3216 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.403555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab69b3b-c059-4092-8abb-d6b2d4de3216 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.408327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43422655-170e-4e06-b9b2-1ed3a761bbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.409086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43422655-170e-4e06-b9b2-1ed3a761bbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.409371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43422655-170e-4e06-b9b2-1ed3a761bbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.409702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43422655-170e-4e06-b9b2-1ed3a761bbf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.410415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43422655-170e-4e06-b9b2-1ed3a761bbf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.412181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43422655-170e-4e06-b9b2-1ed3a761bbf3 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:56.423982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1026/2053] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:56.428425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e640f5-a8e4-4967-8ed8-a56c72b96c0d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.439064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e640f5-a8e4-4967-8ed8-a56c72b96c0d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.443882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365ddf8-8f41-41f1-9fa0-4f0bbc0159bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.444608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ddf8-8f41-41f1-9fa0-4f0bbc0159bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.444886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ddf8-8f41-41f1-9fa0-4f0bbc0159bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.445157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ddf8-8f41-41f1-9fa0-4f0bbc0159bb admin admin] PATH_INFO: `/v3/projects/481cf077e3c34989b6568d11a6544e2b/users/696d407c629d47b482aa4fd1519fdb9d/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.454452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:56.462434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365ddf8-8f41-41f1-9fa0-4f0bbc0159bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=481cf077e3c34989b6568d11a6544e2b, user_id=696d407c629d47b482aa4fd1519fdb9d, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.465585 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365ddf8-8f41-41f1-9fa0-4f0bbc0159bb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:56.479577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1028/2054] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:56 2022] PUT /identity/v3/projects/481cf077e3c34989b6568d11a6544e2b/users/696d407c629d47b482aa4fd1519fdb9d/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:56.491444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a1ce-7452-40d7-841a-54ab003c3502 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.491726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a1ce-7452-40d7-841a-54ab003c3502 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.492003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a1ce-7452-40d7-841a-54ab003c3502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.522742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-5dd7a1ce-7452-40d7-841a-54ab003c3502 None None] MFA Rules not processed for user `696d407c629d47b482aa4fd1519fdb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:56.524119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5dd7a1ce-7452-40d7-841a-54ab003c3502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:56.597351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1027/2055] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:56.602914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365ddf8-8f41-41f1-9fa0-4f0bbc0159bb admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.667678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365ddf8-8f41-41f1-9fa0-4f0bbc0159bb admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.672498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fce652-69bd-44f4-af11-e97f0352b3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.673141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce652-69bd-44f4-af11-e97f0352b3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.673411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce652-69bd-44f4-af11-e97f0352b3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.673688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fce652-69bd-44f4-af11-e97f0352b3a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.674408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fce652-69bd-44f4-af11-e97f0352b3a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.676382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fce652-69bd-44f4-af11-e97f0352b3a5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:56.706660 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1029/2056] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:05:56 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:05:56.711098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a1ce-7452-40d7-841a-54ab003c3502 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.723016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a1ce-7452-40d7-841a-54ab003c3502 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.780185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:56.794051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286283e6-cced-4739-9dbd-9f70041a7721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.794696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286283e6-cced-4739-9dbd-9f70041a7721 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.794973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286283e6-cced-4739-9dbd-9f70041a7721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.795254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286283e6-cced-4739-9dbd-9f70041a7721 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.796037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286283e6-cced-4739-9dbd-9f70041a7721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.798028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286283e6-cced-4739-9dbd-9f70041a7721 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:56.825376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1028/2057] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:05:56 2022] POST /identity/v3/users => generated 299 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:05:56.831009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fce652-69bd-44f4-af11-e97f0352b3a5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.843252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fce652-69bd-44f4-af11-e97f0352b3a5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.848748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5dcc4-c43e-451b-8f09-0fcea16b7285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.849370 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5dcc4-c43e-451b-8f09-0fcea16b7285 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.849657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5dcc4-c43e-451b-8f09-0fcea16b7285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.849944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f5dcc4-c43e-451b-8f09-0fcea16b7285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.850769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5dcc4-c43e-451b-8f09-0fcea16b7285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.852572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f5dcc4-c43e-451b-8f09-0fcea16b7285 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:56.865713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1030/2058] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:05:56.870450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286283e6-cced-4739-9dbd-9f70041a7721 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.880735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286283e6-cced-4739-9dbd-9f70041a7721 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.886752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901976c0-66b9-4298-a940-c6e4d2f2814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.887447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901976c0-66b9-4298-a940-c6e4d2f2814c admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.887721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901976c0-66b9-4298-a940-c6e4d2f2814c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.887989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901976c0-66b9-4298-a940-c6e4d2f2814c admin admin] PATH_INFO: `/v3/projects/9a36702ca4c0446b973478238e5ffc1f/users/c140193680cd4c1597ef61d0e188f253/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.904441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901976c0-66b9-4298-a940-c6e4d2f2814c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a36702ca4c0446b973478238e5ffc1f, user_id=c140193680cd4c1597ef61d0e188f253, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.907150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901976c0-66b9-4298-a940-c6e4d2f2814c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:56.922452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1029/2059] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:56 2022] PUT /identity/v3/projects/9a36702ca4c0446b973478238e5ffc1f/users/c140193680cd4c1597ef61d0e188f253/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:56.926644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5dcc4-c43e-451b-8f09-0fcea16b7285 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:56.937641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f5dcc4-c43e-451b-8f09-0fcea16b7285 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:56.995034 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1a58d-07fb-49b2-a71f-41619344e338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:56.995569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1a58d-07fb-49b2-a71f-41619344e338 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:56.995847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1a58d-07fb-49b2-a71f-41619344e338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:56.996149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1a58d-07fb-49b2-a71f-41619344e338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:56.996893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1a58d-07fb-49b2-a71f-41619344e338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:56.998513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1a58d-07fb-49b2-a71f-41619344e338 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:57.009723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1031/2060] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:05:57.014352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901976c0-66b9-4298-a940-c6e4d2f2814c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:57.025733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901976c0-66b9-4298-a940-c6e4d2f2814c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:57.031792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df8beb-7895-44df-8a56-65b4fd30af79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:57.032586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8beb-7895-44df-8a56-65b4fd30af79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:57.032872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8beb-7895-44df-8a56-65b4fd30af79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:57.033144 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df8beb-7895-44df-8a56-65b4fd30af79 admin admin] PATH_INFO: `/v3/projects/9a36702ca4c0446b973478238e5ffc1f/users/c140193680cd4c1597ef61d0e188f253/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:57.035586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df8beb-7895-44df-8a56-65b4fd30af79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a36702ca4c0446b973478238e5ffc1f, user_id=c140193680cd4c1597ef61d0e188f253, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:57.038625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df8beb-7895-44df-8a56-65b4fd30af79 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:57.053989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1030/2061] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:05:57 2022] PUT /identity/v3/projects/9a36702ca4c0446b973478238e5ffc1f/users/c140193680cd4c1597ef61d0e188f253/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:05:57.065907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae7351-dcc5-4b56-ba20-a14009732953 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:57.066190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae7351-dcc5-4b56-ba20-a14009732953 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:57.066434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae7351-dcc5-4b56-ba20-a14009732953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:57.098325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-f0ae7351-dcc5-4b56-ba20-a14009732953 None None] MFA Rules not processed for user `c140193680cd4c1597ef61d0e188f253`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:05:57.099635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f0ae7351-dcc5-4b56-ba20-a14009732953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:57.149835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:05:57.177902 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1032/2062] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:05:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:05:57.189290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df8beb-7895-44df-8a56-65b4fd30af79 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:57.191176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-34df8beb-7895-44df-8a56-65b4fd30af79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:05:57.276174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df8beb-7895-44df-8a56-65b4fd30af79 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:05:57.279979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3fd75-e482-4bce-a0ea-980dba8d49f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:05:57.280625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3fd75-e482-4bce-a0ea-980dba8d49f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:05:57.280899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3fd75-e482-4bce-a0ea-980dba8d49f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:05:57.281171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3fd75-e482-4bce-a0ea-980dba8d49f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:05:57.349874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3fd75-e482-4bce-a0ea-980dba8d49f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:05:57.351515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3fd75-e482-4bce-a0ea-980dba8d49f9 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:05:57.365853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1031/2063] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:05:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:05:59.950403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae7351-dcc5-4b56-ba20-a14009732953 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:05:59.952236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f0ae7351-dcc5-4b56-ba20-a14009732953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:00.030337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae7351-dcc5-4b56-ba20-a14009732953 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:00.035718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:00.036555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:00.036832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:00.037096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] PATH_INFO: `/v3/users/3a14058bf2284c64b527e9c849c6b72b` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:00.038374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a14058bf2284c64b527e9c849c6b72b)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:00.040326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:00.108566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a14058bf2284c64b527e9c849c6b72b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:00.109500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a14058bf2284c64b527e9c849c6b72b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:00.118975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a14058bf2284c64b527e9c849c6b72b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:00.132454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a14058bf2284c64b527e9c849c6b72b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:00.139487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a14058bf2284c64b527e9c849c6b72b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:00.146610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1033/2064] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:05:59 2022] DELETE /identity/v3/users/3a14058bf2284c64b527e9c849c6b72b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:00.151490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576648ae-8550-4b64-b556-0a794a35bc51 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:00.152081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576648ae-8550-4b64-b556-0a794a35bc51 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:00.152081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576648ae-8550-4b64-b556-0a794a35bc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:00.182485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-576648ae-8550-4b64-b556-0a794a35bc51 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:00.183735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-576648ae-8550-4b64-b556-0a794a35bc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:00.240322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:00.251494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1032/2065] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:00.262221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:00.263715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:00.330496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a9425-484c-4b90-b30c-05d919fff1a5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:00.334739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ff9d4-8db8-4532-a76e-d1edabfc8f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:00.335372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ff9d4-8db8-4532-a76e-d1edabfc8f6e service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:00.335650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ff9d4-8db8-4532-a76e-d1edabfc8f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:00.335924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ff9d4-8db8-4532-a76e-d1edabfc8f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:00.346113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ff9d4-8db8-4532-a76e-d1edabfc8f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:00.347569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ff9d4-8db8-4532-a76e-d1edabfc8f6e service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:00.357654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1034/2066] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:00.721521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576648ae-8550-4b64-b556-0a794a35bc51 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:00.723881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-576648ae-8550-4b64-b556-0a794a35bc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:00.738207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576648ae-8550-4b64-b556-0a794a35bc51 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:00.747518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:00.747518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:00.748000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:00.748000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] PATH_INFO: `/v3/projects/d5b9828957b945c8a68b2cede2237450` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:00.749592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b9828957b945c8a68b2cede2237450)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:00.751603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:00.821583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b9828957b945c8a68b2cede2237450', 'request_id': 'req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:00.833141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b9828957b945c8a68b2cede2237450', 'request_id': 'req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:00.844186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b9828957b945c8a68b2cede2237450', 'request_id': 'req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:00.856701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] The token cache is being invalidate because project d5b9828957b945c8a68b2cede2237450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:00.856998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:00.857762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:00.859436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1033/2067] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:00 2022] DELETE /identity/v3/projects/d5b9828957b945c8a68b2cede2237450 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:00.864538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ff9d4-8db8-4532-a76e-d1edabfc8f6e service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:00.871723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:00.876837 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ff9d4-8db8-4532-a76e-d1edabfc8f6e service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:00.950846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:00.951640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:00.951778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:00.951976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] PATH_INFO: `/v3/users/8d1dfc7e85844e9eb8ccbb2c5b5063b8` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:00.953705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d1dfc7e85844e9eb8ccbb2c5b5063b8)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:00.955805 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:01.022771 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d1dfc7e85844e9eb8ccbb2c5b5063b8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:01.023559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d1dfc7e85844e9eb8ccbb2c5b5063b8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:01.034400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d1dfc7e85844e9eb8ccbb2c5b5063b8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:01.053251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d1dfc7e85844e9eb8ccbb2c5b5063b8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:01.066872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0fb3b6e9-ce30-44e7-aca9-0a129b40b781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d1dfc7e85844e9eb8ccbb2c5b5063b8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:01.074275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1035/2068] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:00 2022] DELETE /identity/v3/users/8d1dfc7e85844e9eb8ccbb2c5b5063b8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:01.478213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:01.479999 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:01.495495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e72a2c-b98a-475d-9f71-84f81ae3f5fc admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:01.596001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:01.596730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:01.597009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:01.597284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] PATH_INFO: `/v3/projects/5a4001a57db14c1f81db972a1b72e1bd` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:01.598533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4001a57db14c1f81db972a1b72e1bd)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:01.600657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:01.663832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4001a57db14c1f81db972a1b72e1bd', 'request_id': 'req-a635431b-b452-48d0-a997-ece48ffac0d3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:01.674024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4001a57db14c1f81db972a1b72e1bd', 'request_id': 'req-a635431b-b452-48d0-a997-ece48ffac0d3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:01.687322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4001a57db14c1f81db972a1b72e1bd', 'request_id': 'req-a635431b-b452-48d0-a997-ece48ffac0d3'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:01.701824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] The token cache is being invalidate because project 5a4001a57db14c1f81db972a1b72e1bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:01.702275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:01.703191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a635431b-b452-48d0-a997-ece48ffac0d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:01.704807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1034/2069] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:01 2022] DELETE /identity/v3/projects/5a4001a57db14c1f81db972a1b72e1bd => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:01.710832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b6072-59d0-4aaf-9e50-e5f38c4ec56a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:01.711121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b6072-59d0-4aaf-9e50-e5f38c4ec56a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:01.711406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b6072-59d0-4aaf-9e50-e5f38c4ec56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:01.732135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-201b6072-59d0-4aaf-9e50-e5f38c4ec56a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:01.733314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-201b6072-59d0-4aaf-9e50-e5f38c4ec56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:01.800279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1036/2070] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:01.808957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724582a5-3e40-4e59-9161-edc305d9a697 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:01.809239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724582a5-3e40-4e59-9161-edc305d9a697 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:01.809513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724582a5-3e40-4e59-9161-edc305d9a697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:01.830509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-724582a5-3e40-4e59-9161-edc305d9a697 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:01.831569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-724582a5-3e40-4e59-9161-edc305d9a697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:01.840218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1035/2071] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:01.845524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b6072-59d0-4aaf-9e50-e5f38c4ec56a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:01.856350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b6072-59d0-4aaf-9e50-e5f38c4ec56a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:01.861396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a6b23-52d2-4ec3-8d23-7daff6ec16e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:01.862028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a6b23-52d2-4ec3-8d23-7daff6ec16e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:01.862314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a6b23-52d2-4ec3-8d23-7daff6ec16e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:01.862582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a6b23-52d2-4ec3-8d23-7daff6ec16e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:01.863304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a6b23-52d2-4ec3-8d23-7daff6ec16e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:01.863640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a6b23-52d2-4ec3-8d23-7daff6ec16e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:01.865393 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a6b23-52d2-4ec3-8d23-7daff6ec16e7 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:01.885143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1037/2072] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:01.886768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724582a5-3e40-4e59-9161-edc305d9a697 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:01.903614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724582a5-3e40-4e59-9161-edc305d9a697 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:01.912770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899557c-e393-4af9-9304-81d2b1680482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:01.913834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899557c-e393-4af9-9304-81d2b1680482 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:01.914307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899557c-e393-4af9-9304-81d2b1680482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:01.914812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899557c-e393-4af9-9304-81d2b1680482 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:01.916009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899557c-e393-4af9-9304-81d2b1680482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:01.919511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899557c-e393-4af9-9304-81d2b1680482 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:01.927732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:01.970813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1036/2073] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:01 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:01.975591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a6b23-52d2-4ec3-8d23-7daff6ec16e7 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:01.986733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a6b23-52d2-4ec3-8d23-7daff6ec16e7 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:01.996185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:02.051869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da677823-eddf-4e2a-ae24-7ccdb595165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:02.052567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da677823-eddf-4e2a-ae24-7ccdb595165b admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:02.052835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da677823-eddf-4e2a-ae24-7ccdb595165b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:02.053110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da677823-eddf-4e2a-ae24-7ccdb595165b admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:02.053812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da677823-eddf-4e2a-ae24-7ccdb595165b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:02.055881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da677823-eddf-4e2a-ae24-7ccdb595165b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:02.090156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1038/2074] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:01 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:02.095333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899557c-e393-4af9-9304-81d2b1680482 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:02.107183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899557c-e393-4af9-9304-81d2b1680482 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:02.112902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b90e0-e020-4b57-829a-8c316ab0d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:02.113597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b90e0-e020-4b57-829a-8c316ab0d033 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:02.113871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b90e0-e020-4b57-829a-8c316ab0d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:02.114220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b90e0-e020-4b57-829a-8c316ab0d033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:02.114934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b90e0-e020-4b57-829a-8c316ab0d033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:02.116675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b90e0-e020-4b57-829a-8c316ab0d033 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:02.128442 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1037/2075] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:02.132805 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da677823-eddf-4e2a-ae24-7ccdb595165b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:02.143394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da677823-eddf-4e2a-ae24-7ccdb595165b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:02.148461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3f83f-bde8-44a7-a6a0-986d6f82249c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:02.149090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f83f-bde8-44a7-a6a0-986d6f82249c admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:02.149457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f83f-bde8-44a7-a6a0-986d6f82249c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:02.149733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f83f-bde8-44a7-a6a0-986d6f82249c admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:02.150609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3f83f-bde8-44a7-a6a0-986d6f82249c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:02.152283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3f83f-bde8-44a7-a6a0-986d6f82249c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:02.165635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1039/2076] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:02.170337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b90e0-e020-4b57-829a-8c316ab0d033 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:02.182077 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b90e0-e020-4b57-829a-8c316ab0d033 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:02.187120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b4a4e-5a37-45f1-9979-2ae8d9dbe4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:02.187814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b4a4e-5a37-45f1-9979-2ae8d9dbe4a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:02.188117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b4a4e-5a37-45f1-9979-2ae8d9dbe4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:02.188442 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b4a4e-5a37-45f1-9979-2ae8d9dbe4a2 admin admin] PATH_INFO: `/v3/projects/d0f6b755d6624595a58cd76cbf296e00/users/b76e6a76bddb48a98d64e7b2dfcc9eff/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:02.205732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b4a4e-5a37-45f1-9979-2ae8d9dbe4a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0f6b755d6624595a58cd76cbf296e00, user_id=b76e6a76bddb48a98d64e7b2dfcc9eff, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:02.208533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57b4a4e-5a37-45f1-9979-2ae8d9dbe4a2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:02.239516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1038/2077] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:02 2022] PUT /identity/v3/projects/d0f6b755d6624595a58cd76cbf296e00/users/b76e6a76bddb48a98d64e7b2dfcc9eff/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:02.250170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e6640-e7d3-4e96-a63e-8a775ec4b35f None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:02.250432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e6640-e7d3-4e96-a63e-8a775ec4b35f None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:02.250716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e6640-e7d3-4e96-a63e-8a775ec4b35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:02.271719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-554e6640-e7d3-4e96-a63e-8a775ec4b35f None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:02.272885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-554e6640-e7d3-4e96-a63e-8a775ec4b35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:02.371960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1040/2078] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:02.400191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b4a4e-5a37-45f1-9979-2ae8d9dbe4a2 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:02.404239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b57b4a4e-5a37-45f1-9979-2ae8d9dbe4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:02.532012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b4a4e-5a37-45f1-9979-2ae8d9dbe4a2 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:02.539901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7404b-84dd-49cc-823b-cfa5f4fd71b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:02.541044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7404b-84dd-49cc-823b-cfa5f4fd71b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:02.541542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7404b-84dd-49cc-823b-cfa5f4fd71b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:02.542019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7404b-84dd-49cc-823b-cfa5f4fd71b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:02.559371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7404b-84dd-49cc-823b-cfa5f4fd71b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:02.561670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7404b-84dd-49cc-823b-cfa5f4fd71b2 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:02.574318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1039/2079] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:04.463230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058899-3f27-449d-8b53-bdcc70df3928 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:04.464124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058899-3f27-449d-8b53-bdcc70df3928 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:04.464124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058899-3f27-449d-8b53-bdcc70df3928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:04.510610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4a058899-3f27-449d-8b53-bdcc70df3928 None None] MFA Rules not processed for user `b76e6a76bddb48a98d64e7b2dfcc9eff`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:04.510833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7404b-84dd-49cc-823b-cfa5f4fd71b2 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:04.512915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4a058899-3f27-449d-8b53-bdcc70df3928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:04.513858 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c0f7404b-84dd-49cc-823b-cfa5f4fd71b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:04.542229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7404b-84dd-49cc-823b-cfa5f4fd71b2 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:04.550906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:04.552613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:04.553772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:04.554443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] PATH_INFO: `/v3/users/be895f2f4b274267a8560b647137c144` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:04.557340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be895f2f4b274267a8560b647137c144)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:04.561511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:04.669496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1041/2080] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:04.671452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a058899-3f27-449d-8b53-bdcc70df3928 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:04.683754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be895f2f4b274267a8560b647137c144'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:04.695177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a058899-3f27-449d-8b53-bdcc70df3928 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:04.704768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3428b-445f-42f9-9c81-07569cbc6e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:04.706334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3428b-445f-42f9-9c81-07569cbc6e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:04.707023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3428b-445f-42f9-9c81-07569cbc6e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:04.707803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3428b-445f-42f9-9c81-07569cbc6e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:04.709250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3428b-445f-42f9-9c81-07569cbc6e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:04.711836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be895f2f4b274267a8560b647137c144'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:04.712594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae3428b-445f-42f9-9c81-07569cbc6e4a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:04.726230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be895f2f4b274267a8560b647137c144'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:04.727975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be895f2f4b274267a8560b647137c144'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:04.729260 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-819f37a2-46f1-4acc-9358-a26170196b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be895f2f4b274267a8560b647137c144'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:04.755227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1040/2081] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:04 2022] DELETE /identity/v3/users/be895f2f4b274267a8560b647137c144 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:04.756894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:04.763112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dc104-fcf8-4e54-8d6f-4128eba72c57 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:04.764117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dc104-fcf8-4e54-8d6f-4128eba72c57 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:04.764647 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dc104-fcf8-4e54-8d6f-4128eba72c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:04.775442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1042/2082] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:04 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:04.778813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3428b-445f-42f9-9c81-07569cbc6e4a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:04.781616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8ae3428b-445f-42f9-9c81-07569cbc6e4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:04.792664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1f7dc104-fcf8-4e54-8d6f-4128eba72c57 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:04.794751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1f7dc104-fcf8-4e54-8d6f-4128eba72c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:04.820177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:04.885051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1041/2083] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:04.895625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dc104-fcf8-4e54-8d6f-4128eba72c57 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:04.897599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1f7dc104-fcf8-4e54-8d6f-4128eba72c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:04.906549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae3428b-445f-42f9-9c81-07569cbc6e4a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:04.911680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67814f1d-f79c-44b2-8f44-00acbe0f9ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:04.912457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67814f1d-f79c-44b2-8f44-00acbe0f9ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:04.912782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67814f1d-f79c-44b2-8f44-00acbe0f9ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:04.913052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67814f1d-f79c-44b2-8f44-00acbe0f9ec5 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:04.913751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67814f1d-f79c-44b2-8f44-00acbe0f9ec5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:04.915751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67814f1d-f79c-44b2-8f44-00acbe0f9ec5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:04.941800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1043/2084] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:04 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:04.946428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67814f1d-f79c-44b2-8f44-00acbe0f9ec5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:04.957405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67814f1d-f79c-44b2-8f44-00acbe0f9ec5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:04.962919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9de79b-7f4c-463e-878b-7c4c86b10e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:04.963721 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9de79b-7f4c-463e-878b-7c4c86b10e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:04.964007 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9de79b-7f4c-463e-878b-7c4c86b10e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:04.964282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9de79b-7f4c-463e-878b-7c4c86b10e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:04.965039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9de79b-7f4c-463e-878b-7c4c86b10e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:04.966731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9de79b-7f4c-463e-878b-7c4c86b10e5b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:04.983552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1044/2085] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:04.985330 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9de79b-7f4c-463e-878b-7c4c86b10e5b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:04.994590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dc104-fcf8-4e54-8d6f-4128eba72c57 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:05.001896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447fca0-bc58-4e29-86cf-677ad9f898cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:05.002399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9de79b-7f4c-463e-878b-7c4c86b10e5b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:05.003214 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447fca0-bc58-4e29-86cf-677ad9f898cd service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:05.003819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447fca0-bc58-4e29-86cf-677ad9f898cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:05.004410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447fca0-bc58-4e29-86cf-677ad9f898cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:05.010470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cec0c8-5898-402b-b411-ffd7ffafefb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:05.011208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cec0c8-5898-402b-b411-ffd7ffafefb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:05.011625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cec0c8-5898-402b-b411-ffd7ffafefb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:05.011854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cec0c8-5898-402b-b411-ffd7ffafefb6 admin admin] PATH_INFO: `/v3/projects/950f0067d6b644249ab9e593d3617344/users/1b0e96724e824414bfce15498d7a1659/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:05.023211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447fca0-bc58-4e29-86cf-677ad9f898cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:05.026130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447fca0-bc58-4e29-86cf-677ad9f898cd service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:05.031730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cec0c8-5898-402b-b411-ffd7ffafefb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=950f0067d6b644249ab9e593d3617344, user_id=1b0e96724e824414bfce15498d7a1659, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:05.034759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cec0c8-5898-402b-b411-ffd7ffafefb6 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:05.048994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1042/2086] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:05.053346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1045/2087] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:04 2022] PUT /identity/v3/projects/950f0067d6b644249ab9e593d3617344/users/1b0e96724e824414bfce15498d7a1659/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:05.058583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447fca0-bc58-4e29-86cf-677ad9f898cd service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:05.074772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447fca0-bc58-4e29-86cf-677ad9f898cd service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:05.190393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1c02a-a431-4fcf-8bda-befb73bf919b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:05.191255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1c02a-a431-4fcf-8bda-befb73bf919b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:05.191796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1c02a-a431-4fcf-8bda-befb73bf919b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:05.192214 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1c02a-a431-4fcf-8bda-befb73bf919b admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:05.195041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1c02a-a431-4fcf-8bda-befb73bf919b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:05.197692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1c02a-a431-4fcf-8bda-befb73bf919b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:05.222719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cec0c8-5898-402b-b411-ffd7ffafefb6 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:05.232731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1043/2088] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:05.238566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cec0c8-5898-402b-b411-ffd7ffafefb6 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:05.248369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae563605-d019-4a42-a307-cc55a519fda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:05.249128 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae563605-d019-4a42-a307-cc55a519fda3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:05.249420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae563605-d019-4a42-a307-cc55a519fda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:05.250643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae563605-d019-4a42-a307-cc55a519fda3 admin admin] PATH_INFO: `/v3/projects/950f0067d6b644249ab9e593d3617344/users/1b0e96724e824414bfce15498d7a1659/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:05.252594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae563605-d019-4a42-a307-cc55a519fda3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=950f0067d6b644249ab9e593d3617344, user_id=1b0e96724e824414bfce15498d7a1659, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:05.257254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae563605-d019-4a42-a307-cc55a519fda3 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:05.281198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1046/2089] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:05 2022] PUT /identity/v3/projects/950f0067d6b644249ab9e593d3617344/users/1b0e96724e824414bfce15498d7a1659/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:05.777352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1c02a-a431-4fcf-8bda-befb73bf919b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:05.779308 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e0b1c02a-a431-4fcf-8bda-befb73bf919b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:05.795422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1c02a-a431-4fcf-8bda-befb73bf919b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:05.902337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:05.904063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:05.905875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:05.905875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] PATH_INFO: `/v3/projects/eacbbfc96d8c4633800d38d3ef0426f6` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:05.908139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eacbbfc96d8c4633800d38d3ef0426f6)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:05.913496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:05.922480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:05.970388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eacbbfc96d8c4633800d38d3ef0426f6', 'request_id': 'req-034ed7c8-724e-4e6a-bf03-56565528de53'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:05.978042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eacbbfc96d8c4633800d38d3ef0426f6', 'request_id': 'req-034ed7c8-724e-4e6a-bf03-56565528de53'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:05.986919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eacbbfc96d8c4633800d38d3ef0426f6', 'request_id': 'req-034ed7c8-724e-4e6a-bf03-56565528de53'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:06.000588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] The token cache is being invalidate because project eacbbfc96d8c4633800d38d3ef0426f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:06.000810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:06.001562 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-034ed7c8-724e-4e6a-bf03-56565528de53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:06.003253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1044/2090] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:05 2022] DELETE /identity/v3/projects/eacbbfc96d8c4633800d38d3ef0426f6 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:06.007257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae563605-d019-4a42-a307-cc55a519fda3 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:06.009044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ae563605-d019-4a42-a307-cc55a519fda3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:06.022090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae563605-d019-4a42-a307-cc55a519fda3 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:06.089131 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:06.089811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:06.090091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:06.090368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] PATH_INFO: `/v3/users/5b76d197cceb4c25bb17c74713ba8eb7` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:06.091593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b76d197cceb4c25bb17c74713ba8eb7)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:06.093639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:06.175810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:06.190070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b76d197cceb4c25bb17c74713ba8eb7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:06.191405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b76d197cceb4c25bb17c74713ba8eb7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:06.203015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b76d197cceb4c25bb17c74713ba8eb7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:06.229488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b76d197cceb4c25bb17c74713ba8eb7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:06.240239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b76d197cceb4c25bb17c74713ba8eb7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:06.248883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1047/2091] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:06 2022] DELETE /identity/v3/users/5b76d197cceb4c25bb17c74713ba8eb7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:06.799623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc1942-cecb-4355-92f9-2b293c980eec None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:06.800245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc1942-cecb-4355-92f9-2b293c980eec None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:06.800245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc1942-cecb-4355-92f9-2b293c980eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:06.855165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-28fc1942-cecb-4355-92f9-2b293c980eec None None] MFA Rules not processed for user `1b0e96724e824414bfce15498d7a1659`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:06.856993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-28fc1942-cecb-4355-92f9-2b293c980eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:06.903943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:06.905465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:07.005972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1045/2092] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:07.013083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc1942-cecb-4355-92f9-2b293c980eec None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:07.015138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-28fc1942-cecb-4355-92f9-2b293c980eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:07.119712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd526c-7615-4718-be05-82756ec40564 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:07.141167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:07.141962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:07.143031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:07.144327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] PATH_INFO: `/v3/projects/b498ea92e45047399fe857777a43bb8c` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:07.147686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b498ea92e45047399fe857777a43bb8c)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:07.153476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:07.224361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc1942-cecb-4355-92f9-2b293c980eec None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:07.266212 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b498ea92e45047399fe857777a43bb8c', 'request_id': 'req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:07.274730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b498ea92e45047399fe857777a43bb8c', 'request_id': 'req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:07.288979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b498ea92e45047399fe857777a43bb8c', 'request_id': 'req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:07.297024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:07.316192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] The token cache is being invalidate because project b498ea92e45047399fe857777a43bb8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:07.317229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:07.318668 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b3335e97-3cac-4f64-b78f-748fc3ebdc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:07.321892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1048/2093] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:06 2022] DELETE /identity/v3/projects/b498ea92e45047399fe857777a43bb8c => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:07.327517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e618ea8-2981-4031-b9f0-6af20aeb07a3 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:07.327812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e618ea8-2981-4031-b9f0-6af20aeb07a3 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:07.328648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e618ea8-2981-4031-b9f0-6af20aeb07a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:07.334216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ed734-b836-4451-a6f4-9f87bad27337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:07.334857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ed734-b836-4451-a6f4-9f87bad27337 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:07.335159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ed734-b836-4451-a6f4-9f87bad27337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:07.335417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ed734-b836-4451-a6f4-9f87bad27337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:07.337680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ec2ed734-b836-4451-a6f4-9f87bad27337 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:07.370848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6e618ea8-2981-4031-b9f0-6af20aeb07a3 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:07.372176 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6e618ea8-2981-4031-b9f0-6af20aeb07a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:07.433647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:07.439666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ed734-b836-4451-a6f4-9f87bad27337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:07.442203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ed734-b836-4451-a6f4-9f87bad27337 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:07.457332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1046/2094] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:07.470922 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1049/2095] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:07.473213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81058-0da9-4407-b554-f348e243eec5 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:07.473789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81058-0da9-4407-b554-f348e243eec5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:07.474272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf81058-0da9-4407-b554-f348e243eec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:07.497709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-daf81058-0da9-4407-b554-f348e243eec5 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:07.499095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-daf81058-0da9-4407-b554-f348e243eec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:07.508088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1047/2096] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:07.514164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e618ea8-2981-4031-b9f0-6af20aeb07a3 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:07.525694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e618ea8-2981-4031-b9f0-6af20aeb07a3 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:07.531703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2c7c8-de18-418c-bb26-8565437a0ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:07.532475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2c7c8-de18-418c-bb26-8565437a0ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:07.533000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2c7c8-de18-418c-bb26-8565437a0ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:07.533293 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2c7c8-de18-418c-bb26-8565437a0ef5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:07.534047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2c7c8-de18-418c-bb26-8565437a0ef5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:07.534359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2c7c8-de18-418c-bb26-8565437a0ef5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:07.536364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2c7c8-de18-418c-bb26-8565437a0ef5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:07.550599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1050/2097] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:06:07.556430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf81058-0da9-4407-b554-f348e243eec5 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:07.580176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf81058-0da9-4407-b554-f348e243eec5 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:07.589837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1059-a4c4-42f6-b3b2-98762f08909b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:07.590905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1059-a4c4-42f6-b3b2-98762f08909b admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:07.591403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1059-a4c4-42f6-b3b2-98762f08909b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:07.591906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b1059-a4c4-42f6-b3b2-98762f08909b admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:07.593285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b1059-a4c4-42f6-b3b2-98762f08909b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:07.597211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b1059-a4c4-42f6-b3b2-98762f08909b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:07.655049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1048/2098] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:07 2022] POST /identity/v3/projects => generated 391 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:07.662267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2c7c8-de18-418c-bb26-8565437a0ef5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:07.686500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2c7c8-de18-418c-bb26-8565437a0ef5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:07.836673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b72fc8-bce8-4c38-b98e-8f652e0b0a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:07.837268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b72fc8-bce8-4c38-b98e-8f652e0b0a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:07.837648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b72fc8-bce8-4c38-b98e-8f652e0b0a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:07.837803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b72fc8-bce8-4c38-b98e-8f652e0b0a9a admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:07.838555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b72fc8-bce8-4c38-b98e-8f652e0b0a9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:07.840897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b72fc8-bce8-4c38-b98e-8f652e0b0a9a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:07.894979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1051/2099] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:07 2022] POST /identity/v3/users => generated 296 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:07.901045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1059-a4c4-42f6-b3b2-98762f08909b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:07.930839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b1059-a4c4-42f6-b3b2-98762f08909b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:07.945488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10d8d7-2821-407f-80b3-0580752a44e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:07.946628 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10d8d7-2821-407f-80b3-0580752a44e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:07.947101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10d8d7-2821-407f-80b3-0580752a44e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:07.947575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10d8d7-2821-407f-80b3-0580752a44e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:07.948974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10d8d7-2821-407f-80b3-0580752a44e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:07.952644 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10d8d7-2821-407f-80b3-0580752a44e2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:07.976233 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1049/2100] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:07.983591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b72fc8-bce8-4c38-b98e-8f652e0b0a9a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:08.026390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b72fc8-bce8-4c38-b98e-8f652e0b0a9a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:08.049700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d488cc-d0b7-4691-8d6c-0c3820aa4281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:08.053004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d488cc-d0b7-4691-8d6c-0c3820aa4281 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:08.054880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d488cc-d0b7-4691-8d6c-0c3820aa4281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:08.056262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d488cc-d0b7-4691-8d6c-0c3820aa4281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:08.057798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d488cc-d0b7-4691-8d6c-0c3820aa4281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:08.061050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d488cc-d0b7-4691-8d6c-0c3820aa4281 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:08.163909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1052/2101] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:08.166760 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10d8d7-2821-407f-80b3-0580752a44e2 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:08.195284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10d8d7-2821-407f-80b3-0580752a44e2 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:08.218154 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5909f-e091-4662-8993-99dd21167250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:08.219983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5909f-e091-4662-8993-99dd21167250 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:08.220570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5909f-e091-4662-8993-99dd21167250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:08.221237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5909f-e091-4662-8993-99dd21167250 admin admin] PATH_INFO: `/v3/projects/65aa4dfcde1e4b5ea3232b8defd5711b/users/1f490d9a4d264b838703e529d8971b81/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:08.266417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5909f-e091-4662-8993-99dd21167250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65aa4dfcde1e4b5ea3232b8defd5711b, user_id=1f490d9a4d264b838703e529d8971b81, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:08.272828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5909f-e091-4662-8993-99dd21167250 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:08.319801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d488cc-d0b7-4691-8d6c-0c3820aa4281 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:08.323248 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-60d488cc-d0b7-4691-8d6c-0c3820aa4281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:08.324361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1050/2102] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:08 2022] PUT /identity/v3/projects/65aa4dfcde1e4b5ea3232b8defd5711b/users/1f490d9a4d264b838703e529d8971b81/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:08.347802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f67c76-0827-49c2-bea5-a42da159a3ea None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:08.348360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f67c76-0827-49c2-bea5-a42da159a3ea None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:08.348515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f67c76-0827-49c2-bea5-a42da159a3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:08.434022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-37f67c76-0827-49c2-bea5-a42da159a3ea None None] MFA Rules not processed for user `1f490d9a4d264b838703e529d8971b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:08.435338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-37f67c76-0827-49c2-bea5-a42da159a3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:08.506250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d488cc-d0b7-4691-8d6c-0c3820aa4281 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:08.630696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1051/2103] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:08.640518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f67c76-0827-49c2-bea5-a42da159a3ea None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:08.652601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afe37c-82e8-4bdb-be09-c30bb9da4c4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:08.653785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afe37c-82e8-4bdb-be09-c30bb9da4c4c service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:08.654324 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afe37c-82e8-4bdb-be09-c30bb9da4c4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:08.654799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afe37c-82e8-4bdb-be09-c30bb9da4c4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:08.659400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-15afe37c-82e8-4bdb-be09-c30bb9da4c4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:08.695111 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:08.728573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:08.804202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f67c76-0827-49c2-bea5-a42da159a3ea None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:08.812928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b111d-136b-4f1f-887e-a146bf0471ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:08.814098 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b111d-136b-4f1f-887e-a146bf0471ff admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:08.814640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b111d-136b-4f1f-887e-a146bf0471ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:08.815151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b111d-136b-4f1f-887e-a146bf0471ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:08.816459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b111d-136b-4f1f-887e-a146bf0471ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:08.820109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b111d-136b-4f1f-887e-a146bf0471ff admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:08.823323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afe37c-82e8-4bdb-be09-c30bb9da4c4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:08.827298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afe37c-82e8-4bdb-be09-c30bb9da4c4c service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:08.859649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1053/2104] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:08.869960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1052/2105] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:08 2022] POST /identity/v3/projects => generated 393 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:08.874976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afe37c-82e8-4bdb-be09-c30bb9da4c4c service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:08.896282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afe37c-82e8-4bdb-be09-c30bb9da4c4c service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:08.980616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d60ae-ef82-4f49-9645-623568f754c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:08.981271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d60ae-ef82-4f49-9645-623568f754c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:08.981552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d60ae-ef82-4f49-9645-623568f754c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:08.981830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d60ae-ef82-4f49-9645-623568f754c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:08.982545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d60ae-ef82-4f49-9645-623568f754c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:08.984644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d60ae-ef82-4f49-9645-623568f754c1 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:09.022144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1054/2106] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:08 2022] POST /identity/v3/users => generated 303 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:09.023735 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b111d-136b-4f1f-887e-a146bf0471ff admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:09.050009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b111d-136b-4f1f-887e-a146bf0471ff admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:09.058323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3d926-be3e-4f6f-a02d-69283913e6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:09.059101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3d926-be3e-4f6f-a02d-69283913e6ab admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:09.059367 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3d926-be3e-4f6f-a02d-69283913e6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:09.059679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3d926-be3e-4f6f-a02d-69283913e6ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:09.060490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3d926-be3e-4f6f-a02d-69283913e6ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:09.062431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3d926-be3e-4f6f-a02d-69283913e6ab admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:09.081830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1053/2107] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:09.087931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d60ae-ef82-4f49-9645-623568f754c1 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:09.107604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d60ae-ef82-4f49-9645-623568f754c1 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:09.116099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e4eb-741d-43c0-8c3a-b9f87e07b472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:09.117289 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e4eb-741d-43c0-8c3a-b9f87e07b472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:09.117793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e4eb-741d-43c0-8c3a-b9f87e07b472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:09.118651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962e4eb-741d-43c0-8c3a-b9f87e07b472 admin admin] PATH_INFO: `/v3/projects/8cf6275550fe458882cb0bc0bf6b3565/users/9faf494951fa469292a991e0c21d15f4/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:09.149379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962e4eb-741d-43c0-8c3a-b9f87e07b472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf6275550fe458882cb0bc0bf6b3565, user_id=9faf494951fa469292a991e0c21d15f4, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:09.154324 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962e4eb-741d-43c0-8c3a-b9f87e07b472 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:09.187507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1055/2108] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:09 2022] PUT /identity/v3/projects/8cf6275550fe458882cb0bc0bf6b3565/users/9faf494951fa469292a991e0c21d15f4/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:09.196533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3d926-be3e-4f6f-a02d-69283913e6ab admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:09.220120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3d926-be3e-4f6f-a02d-69283913e6ab admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:09.326584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08bd70-072a-4f88-9354-a1bfc3886770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:09.327268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08bd70-072a-4f88-9354-a1bfc3886770 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:09.327569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08bd70-072a-4f88-9354-a1bfc3886770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:09.327877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08bd70-072a-4f88-9354-a1bfc3886770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:09.328629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08bd70-072a-4f88-9354-a1bfc3886770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:09.330299 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08bd70-072a-4f88-9354-a1bfc3886770 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:09.343128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1054/2109] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:09 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:09.347690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e4eb-741d-43c0-8c3a-b9f87e07b472 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:09.365933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962e4eb-741d-43c0-8c3a-b9f87e07b472 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:09.374511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d693e-f158-43ce-814b-252b9598da20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:09.375537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d693e-f158-43ce-814b-252b9598da20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:09.375958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d693e-f158-43ce-814b-252b9598da20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:09.376696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d693e-f158-43ce-814b-252b9598da20 admin admin] PATH_INFO: `/v3/projects/8cf6275550fe458882cb0bc0bf6b3565/users/9faf494951fa469292a991e0c21d15f4/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:09.379827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d693e-f158-43ce-814b-252b9598da20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cf6275550fe458882cb0bc0bf6b3565, user_id=9faf494951fa469292a991e0c21d15f4, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:09.384141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d693e-f158-43ce-814b-252b9598da20 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:09.405912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1056/2110] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:09 2022] PUT /identity/v3/projects/8cf6275550fe458882cb0bc0bf6b3565/users/9faf494951fa469292a991e0c21d15f4/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:09.415116 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08bd70-072a-4f88-9354-a1bfc3886770 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:09.417025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87779339-d78f-4b81-99ad-8f2055657849 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:09.417613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87779339-d78f-4b81-99ad-8f2055657849 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:09.418090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87779339-d78f-4b81-99ad-8f2055657849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:09.418294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-db08bd70-072a-4f88-9354-a1bfc3886770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:09.466202 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-87779339-d78f-4b81-99ad-8f2055657849 None None] MFA Rules not processed for user `9faf494951fa469292a991e0c21d15f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:09.469452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-87779339-d78f-4b81-99ad-8f2055657849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:09.502320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08bd70-072a-4f88-9354-a1bfc3886770 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:09.566885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:09.582129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff7c9a-8969-436a-9151-e67e842a0ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:09.583336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff7c9a-8969-436a-9151-e67e842a0ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:09.583662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff7c9a-8969-436a-9151-e67e842a0ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:09.583969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff7c9a-8969-436a-9151-e67e842a0ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:09.586459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e6ff7c9a-8969-436a-9151-e67e842a0ee0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:09.602555 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:09.610019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1057/2111] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:09.630192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87779339-d78f-4b81-99ad-8f2055657849 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:09.642516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87779339-d78f-4b81-99ad-8f2055657849 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:09.647566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9112c1-e0ad-455e-8931-fa67c178becc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:09.648361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9112c1-e0ad-455e-8931-fa67c178becc service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:09.648843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9112c1-e0ad-455e-8931-fa67c178becc service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:09.648843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9112c1-e0ad-455e-8931-fa67c178becc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:09.661558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9112c1-e0ad-455e-8931-fa67c178becc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:09.663201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9112c1-e0ad-455e-8931-fa67c178becc service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:09.677132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1058/2112] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:09.703123 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff7c9a-8969-436a-9151-e67e842a0ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:09.704643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff7c9a-8969-436a-9151-e67e842a0ee0 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:09.715870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1055/2113] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:09.751380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9112c1-e0ad-455e-8931-fa67c178becc service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:09.760383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9112c1-e0ad-455e-8931-fa67c178becc service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:09.764112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24b394-c2c2-45ca-b4e5-eef5ebc00733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:09.764785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24b394-c2c2-45ca-b4e5-eef5ebc00733 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:09.765067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24b394-c2c2-45ca-b4e5-eef5ebc00733 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:09.765346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24b394-c2c2-45ca-b4e5-eef5ebc00733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:09.843091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24b394-c2c2-45ca-b4e5-eef5ebc00733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:09.844765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24b394-c2c2-45ca-b4e5-eef5ebc00733 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:09.855573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1059/2114] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:09 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:15.179898 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff7c9a-8969-436a-9151-e67e842a0ee0 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:15.325912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff7c9a-8969-436a-9151-e67e842a0ee0 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:15.334507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:15.335231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:15.335521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:15.335794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] PATH_INFO: `/v3/users/1f490d9a4d264b838703e529d8971b81` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:15.338839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f490d9a4d264b838703e529d8971b81)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:15.343458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:15.450725 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f490d9a4d264b838703e529d8971b81'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:15.465483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f490d9a4d264b838703e529d8971b81'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:15.473515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f490d9a4d264b838703e529d8971b81'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:15.474289 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f490d9a4d264b838703e529d8971b81'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:15.474899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f490d9a4d264b838703e529d8971b81'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:15.493545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1056/2115] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:15 2022] DELETE /identity/v3/users/1f490d9a4d264b838703e529d8971b81 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:15.499141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef0226-77cd-4d3b-9c6f-b182ad67ecce None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:15.500043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef0226-77cd-4d3b-9c6f-b182ad67ecce None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:15.500043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef0226-77cd-4d3b-9c6f-b182ad67ecce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:15.523710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c0ef0226-77cd-4d3b-9c6f-b182ad67ecce None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:15.525029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c0ef0226-77cd-4d3b-9c6f-b182ad67ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:15.602368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1060/2116] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:15.614776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:15.617683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:15.639481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:15.717495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198e9b5-233e-4eb5-ac84-093991ea01d6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:15.722202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0aef14-e4ac-46f6-8e00-c56a72be4090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:15.722914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aef14-e4ac-46f6-8e00-c56a72be4090 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:15.723399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aef14-e4ac-46f6-8e00-c56a72be4090 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:15.723692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0aef14-e4ac-46f6-8e00-c56a72be4090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:15.736284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0aef14-e4ac-46f6-8e00-c56a72be4090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:15.738050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0aef14-e4ac-46f6-8e00-c56a72be4090 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:15.750991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1057/2117] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:16.127148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef0226-77cd-4d3b-9c6f-b182ad67ecce None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:16.128910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c0ef0226-77cd-4d3b-9c6f-b182ad67ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:16.144730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef0226-77cd-4d3b-9c6f-b182ad67ecce None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:16.150023 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:16.150702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:16.150983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:16.151263 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] PATH_INFO: `/v3/projects/65aa4dfcde1e4b5ea3232b8defd5711b` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:16.152562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] RBAC: Authorizing `identity:delete_project(project_id=65aa4dfcde1e4b5ea3232b8defd5711b)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:16.154747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:16.224210 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65aa4dfcde1e4b5ea3232b8defd5711b', 'request_id': 'req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.233428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65aa4dfcde1e4b5ea3232b8defd5711b', 'request_id': 'req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.247626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65aa4dfcde1e4b5ea3232b8defd5711b', 'request_id': 'req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.264053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] The token cache is being invalidate because project 65aa4dfcde1e4b5ea3232b8defd5711b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:16.264053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.264938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.266771 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1061/2118] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:16 2022] DELETE /identity/v3/projects/65aa4dfcde1e4b5ea3232b8defd5711b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:16.271042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0aef14-e4ac-46f6-8e00-c56a72be4090 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:16.282904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0aef14-e4ac-46f6-8e00-c56a72be4090 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:16.364422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:16.365433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:16.365876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:16.366326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] PATH_INFO: `/v3/users/9faf494951fa469292a991e0c21d15f4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:16.368549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9faf494951fa469292a991e0c21d15f4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:16.371817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:16.455846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9faf494951fa469292a991e0c21d15f4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.482635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9faf494951fa469292a991e0c21d15f4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.507499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9faf494951fa469292a991e0c21d15f4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.508788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9faf494951fa469292a991e0c21d15f4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.515953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-7b85b741-7fbe-4078-9afe-9fb39889abfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9faf494951fa469292a991e0c21d15f4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.519048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:16.523475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:16.562456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1058/2119] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:16 2022] DELETE /identity/v3/users/9faf494951fa469292a991e0c21d15f4 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:16.580747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0cdc1-a65b-45b3-b36e-3111db4bc39e admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:16.622776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:16.772048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:16.773139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:16.773510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:16.773900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] PATH_INFO: `/v3/users/696d407c629d47b482aa4fd1519fdb9d` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:16.775857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=696d407c629d47b482aa4fd1519fdb9d)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:16.778224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:16.886757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '696d407c629d47b482aa4fd1519fdb9d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.887663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '696d407c629d47b482aa4fd1519fdb9d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.899665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '696d407c629d47b482aa4fd1519fdb9d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.922262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '696d407c629d47b482aa4fd1519fdb9d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.935041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '696d407c629d47b482aa4fd1519fdb9d'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:16.949375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1062/2120] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:16 2022] DELETE /identity/v3/users/696d407c629d47b482aa4fd1519fdb9d => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:16.955857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c3ff8-30ab-4ff5-86dd-43e5b897cd38 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:16.956384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c3ff8-30ab-4ff5-86dd-43e5b897cd38 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:16.956869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c3ff8-30ab-4ff5-86dd-43e5b897cd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:16.982784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a59c3ff8-30ab-4ff5-86dd-43e5b897cd38 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:16.984275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a59c3ff8-30ab-4ff5-86dd-43e5b897cd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:17.025160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:17.065927 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:17.068217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:17.088073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1059/2121] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:17.101225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c3ff8-30ab-4ff5-86dd-43e5b897cd38 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:17.103919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a59c3ff8-30ab-4ff5-86dd-43e5b897cd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:17.192185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4f2e7-ba4a-40a2-8452-05904151e0b4 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:17.192596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c3ff8-30ab-4ff5-86dd-43e5b897cd38 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:17.196673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efb821-209a-403a-a64e-b22be9d8e98e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:17.197242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:17.197512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efb821-209a-403a-a64e-b22be9d8e98e service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:17.197864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efb821-209a-403a-a64e-b22be9d8e98e service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:17.198062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:17.198268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efb821-209a-403a-a64e-b22be9d8e98e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:17.198425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:17.198601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] PATH_INFO: `/v3/projects/8cf6275550fe458882cb0bc0bf6b3565` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:17.199818 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cf6275550fe458882cb0bc0bf6b3565)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:17.201817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:17.210508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efb821-209a-403a-a64e-b22be9d8e98e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:17.211979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efb821-209a-403a-a64e-b22be9d8e98e service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:17.224028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1060/2122] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:17.245171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:17.268850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cf6275550fe458882cb0bc0bf6b3565', 'request_id': 'req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:17.277526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cf6275550fe458882cb0bc0bf6b3565', 'request_id': 'req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:17.288917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cf6275550fe458882cb0bc0bf6b3565', 'request_id': 'req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:17.305583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] The token cache is being invalidate because project 8cf6275550fe458882cb0bc0bf6b3565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:17.305962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:17.306941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c18126f8-4d04-4ff3-9732-2d2a8cdca8c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:17.309541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1063/2123] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:17 2022] DELETE /identity/v3/projects/8cf6275550fe458882cb0bc0bf6b3565 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:17.316238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ff53c-2577-465e-aea9-4049b1d1563a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:17.317287 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ff53c-2577-465e-aea9-4049b1d1563a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:17.317287 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ff53c-2577-465e-aea9-4049b1d1563a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:17.350288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d87ff53c-2577-465e-aea9-4049b1d1563a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:17.352333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d87ff53c-2577-465e-aea9-4049b1d1563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:17.455075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1061/2124] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:17.464751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb374a-1aeb-4b42-aace-6f686c23f9eb None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:17.465020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb374a-1aeb-4b42-aace-6f686c23f9eb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:17.465292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb374a-1aeb-4b42-aace-6f686c23f9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:17.524871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4adb374a-1aeb-4b42-aace-6f686c23f9eb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:17.526166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4adb374a-1aeb-4b42-aace-6f686c23f9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:17.536794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1064/2125] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:17.543381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ff53c-2577-465e-aea9-4049b1d1563a None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:17.562675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ff53c-2577-465e-aea9-4049b1d1563a None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:17.573156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390095c-96bb-4e26-ba56-6b1743fbef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:17.573820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390095c-96bb-4e26-ba56-6b1743fbef17 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:17.574117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390095c-96bb-4e26-ba56-6b1743fbef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:17.574417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390095c-96bb-4e26-ba56-6b1743fbef17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:17.575277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390095c-96bb-4e26-ba56-6b1743fbef17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:17.575916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390095c-96bb-4e26-ba56-6b1743fbef17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:17.578140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390095c-96bb-4e26-ba56-6b1743fbef17 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:17.612329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1062/2126] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:17.618819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb374a-1aeb-4b42-aace-6f686c23f9eb None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:17.635323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb374a-1aeb-4b42-aace-6f686c23f9eb None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:17.641784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3f168-028d-4027-9363-dcd20361f70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:17.642435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3f168-028d-4027-9363-dcd20361f70e admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:17.642875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3f168-028d-4027-9363-dcd20361f70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:17.643328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3f168-028d-4027-9363-dcd20361f70e admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:17.644043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3f168-028d-4027-9363-dcd20361f70e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:17.646565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3f168-028d-4027-9363-dcd20361f70e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:17.691932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1065/2127] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:17 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:17.698226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390095c-96bb-4e26-ba56-6b1743fbef17 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:17.717911 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3f168-028d-4027-9363-dcd20361f70e admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:17.720847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390095c-96bb-4e26-ba56-6b1743fbef17 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:17.721966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e6a3f168-028d-4027-9363-dcd20361f70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:17.864847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8435-c50f-4014-8d25-1bb75588bbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:17.867801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8435-c50f-4014-8d25-1bb75588bbba admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:17.867999 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8435-c50f-4014-8d25-1bb75588bbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:17.869189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f8435-c50f-4014-8d25-1bb75588bbba admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:17.870989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8435-c50f-4014-8d25-1bb75588bbba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:17.877550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f8435-c50f-4014-8d25-1bb75588bbba admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:17.910517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:17.946604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1063/2128] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:17 2022] POST /identity/v3/users => generated 288 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:17.950873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8435-c50f-4014-8d25-1bb75588bbba admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:17.958626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3f168-028d-4027-9363-dcd20361f70e admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:17.970245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:17.971100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:17.971537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:17.971969 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] PATH_INFO: `/v3/projects/481cf077e3c34989b6568d11a6544e2b` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:17.973542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=481cf077e3c34989b6568d11a6544e2b)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:17.976173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:17.987373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f8435-c50f-4014-8d25-1bb75588bbba admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:18.006316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23355627-6dd9-4d58-a9e6-c88373eda4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:18.006965 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23355627-6dd9-4d58-a9e6-c88373eda4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:18.007244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23355627-6dd9-4d58-a9e6-c88373eda4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:18.007510 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23355627-6dd9-4d58-a9e6-c88373eda4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:18.008257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23355627-6dd9-4d58-a9e6-c88373eda4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:18.009939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23355627-6dd9-4d58-a9e6-c88373eda4ce admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:18.040618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1064/2129] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:17 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:18.046085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23355627-6dd9-4d58-a9e6-c88373eda4ce admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:18.058247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23355627-6dd9-4d58-a9e6-c88373eda4ce admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:18.081356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '481cf077e3c34989b6568d11a6544e2b', 'request_id': 'req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:18.105169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '481cf077e3c34989b6568d11a6544e2b', 'request_id': 'req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:18.124088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '481cf077e3c34989b6568d11a6544e2b', 'request_id': 'req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:18.172201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] The token cache is being invalidate because project 481cf077e3c34989b6568d11a6544e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:18.172672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:18.173839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:18.179622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1066/2130] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:17 2022] DELETE /identity/v3/projects/481cf077e3c34989b6568d11a6544e2b => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:18.179730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40321b7a-b42a-4379-b3df-973bb799604f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:18.179730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40321b7a-b42a-4379-b3df-973bb799604f admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:18.179730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40321b7a-b42a-4379-b3df-973bb799604f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:18.179730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40321b7a-b42a-4379-b3df-973bb799604f admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:18.181484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40321b7a-b42a-4379-b3df-973bb799604f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:18.181642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:18.184155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:18.184443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40321b7a-b42a-4379-b3df-973bb799604f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:18.207576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1065/2131] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:18.209748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dfef4-700b-4ebd-a347-1ee9709b82dd admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:18.214013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40321b7a-b42a-4379-b3df-973bb799604f admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:18.216143 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-40321b7a-b42a-4379-b3df-973bb799604f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:18.216958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:18.217828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:18.218277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:18.218711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] PATH_INFO: `/v3/users/c140193680cd4c1597ef61d0e188f253` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:18.220214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c140193680cd4c1597ef61d0e188f253)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:18.223163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:18.234576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40321b7a-b42a-4379-b3df-973bb799604f admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:18.241260 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d59b34-ec09-4944-8016-f8cddc7962b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:18.241970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d59b34-ec09-4944-8016-f8cddc7962b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:18.242249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d59b34-ec09-4944-8016-f8cddc7962b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:18.242958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d59b34-ec09-4944-8016-f8cddc7962b5 admin admin] PATH_INFO: `/v3/projects/9d64b692cd8f4d058205c48b46abef31/users/80c525991a06473f889dd830c8d842db/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:18.247221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:18.267695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d59b34-ec09-4944-8016-f8cddc7962b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d64b692cd8f4d058205c48b46abef31, user_id=80c525991a06473f889dd830c8d842db, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:18.271580 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d59b34-ec09-4944-8016-f8cddc7962b5 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:18.293295 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1066/2132] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:18 2022] PUT /identity/v3/projects/9d64b692cd8f4d058205c48b46abef31/users/80c525991a06473f889dd830c8d842db/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:18.304041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4759f2-b833-4b4f-900a-210bad26c536 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:18.304351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4759f2-b833-4b4f-900a-210bad26c536 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:18.304619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4759f2-b833-4b4f-900a-210bad26c536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:18.306495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c140193680cd4c1597ef61d0e188f253'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:18.307484 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c140193680cd4c1597ef61d0e188f253'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:18.318591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c140193680cd4c1597ef61d0e188f253'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:18.338776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c140193680cd4c1597ef61d0e188f253'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:18.347260 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1a4759f2-b833-4b4f-900a-210bad26c536 None None] MFA Rules not processed for user `80c525991a06473f889dd830c8d842db`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:18.349365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1a4759f2-b833-4b4f-900a-210bad26c536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:18.351233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c140193680cd4c1597ef61d0e188f253'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:18.361295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1067/2133] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:18 2022] DELETE /identity/v3/users/c140193680cd4c1597ef61d0e188f253 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:18.457329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1067/2134] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:18.470217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:18.473356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:18.578844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998dc5c-406c-4f37-b4e7-4949b8920689 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:18.583637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af01a2e-7c19-499e-97da-4759e64d53cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:18.584434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af01a2e-7c19-499e-97da-4759e64d53cf service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:18.584594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af01a2e-7c19-499e-97da-4759e64d53cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:18.584855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af01a2e-7c19-499e-97da-4759e64d53cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:18.689909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af01a2e-7c19-499e-97da-4759e64d53cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:18.691637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af01a2e-7c19-499e-97da-4759e64d53cf service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:18.703526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1068/2135] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:19.032868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4759f2-b833-4b4f-900a-210bad26c536 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:19.034641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1a4759f2-b833-4b4f-900a-210bad26c536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:19.107896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:19.134616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4759f2-b833-4b4f-900a-210bad26c536 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:19.255594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:19.256721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:19.257203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:19.257876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] PATH_INFO: `/v3/users/b93fd3b624134a41ba7e2ad7564c363e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:19.260060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b93fd3b624134a41ba7e2ad7564c363e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:19.263712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:19.326707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af01a2e-7c19-499e-97da-4759e64d53cf service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:19.328688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1af01a2e-7c19-499e-97da-4759e64d53cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:19.357077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af01a2e-7c19-499e-97da-4759e64d53cf service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:19.370113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b93fd3b624134a41ba7e2ad7564c363e'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:19.378882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:19.380143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:19.382093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:19.382756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] PATH_INFO: `/v3/projects/9a36702ca4c0446b973478238e5ffc1f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:19.386405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a36702ca4c0446b973478238e5ffc1f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:19.392210 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:19.407342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b93fd3b624134a41ba7e2ad7564c363e'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:19.430534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b93fd3b624134a41ba7e2ad7564c363e'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:19.432477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b93fd3b624134a41ba7e2ad7564c363e'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:19.433733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-06248992-7847-4a69-9140-2e7f2471e795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b93fd3b624134a41ba7e2ad7564c363e'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:19.473554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1068/2136] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:19 2022] DELETE /identity/v3/users/b93fd3b624134a41ba7e2ad7564c363e => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:19.478453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f429c04-9b01-4ac9-a169-77951c7d4317 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:19.478721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f429c04-9b01-4ac9-a169-77951c7d4317 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:19.478906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f429c04-9b01-4ac9-a169-77951c7d4317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:19.500656 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:19.524361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-2f429c04-9b01-4ac9-a169-77951c7d4317 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:19.525883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a36702ca4c0446b973478238e5ffc1f', 'request_id': 'req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:19.526201 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2f429c04-9b01-4ac9-a169-77951c7d4317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:19.548798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a36702ca4c0446b973478238e5ffc1f', 'request_id': 'req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:19.565446 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a36702ca4c0446b973478238e5ffc1f', 'request_id': 'req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:19.594471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] The token cache is being invalidate because project 9a36702ca4c0446b973478238e5ffc1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:19.595142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:19.596056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a796a218-a9e0-4dcc-8881-5ba77f7f8b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:19.600653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1069/2137] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:19 2022] DELETE /identity/v3/projects/9a36702ca4c0446b973478238e5ffc1f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:19.611509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc99f1-5a30-4bea-b334-0b7872450f76 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:19.612340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc99f1-5a30-4bea-b334-0b7872450f76 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:19.613137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccc99f1-5a30-4bea-b334-0b7872450f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:19.695491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1069/2138] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:19.700515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-cccc99f1-5a30-4bea-b334-0b7872450f76 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:19.705055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-cccc99f1-5a30-4bea-b334-0b7872450f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:19.713437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f429c04-9b01-4ac9-a169-77951c7d4317 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:19.718670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2f429c04-9b01-4ac9-a169-77951c7d4317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:19.729311 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1070/2139] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:19.749515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab5406-8ce0-4e80-886c-dd3793bc837d None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:19.750143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab5406-8ce0-4e80-886c-dd3793bc837d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:19.750425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab5406-8ce0-4e80-886c-dd3793bc837d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:19.765217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:19.817736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-b4ab5406-8ce0-4e80-886c-dd3793bc837d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:19.824084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b4ab5406-8ce0-4e80-886c-dd3793bc837d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:19.842831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1071/2140] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:19.854567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab5406-8ce0-4e80-886c-dd3793bc837d None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:19.868702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab5406-8ce0-4e80-886c-dd3793bc837d None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:19.880717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73815a38-5946-4339-83ef-f3309bdcbf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:19.882795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73815a38-5946-4339-83ef-f3309bdcbf4a admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:19.883086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73815a38-5946-4339-83ef-f3309bdcbf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:19.883359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73815a38-5946-4339-83ef-f3309bdcbf4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:19.884371 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73815a38-5946-4339-83ef-f3309bdcbf4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:19.884497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73815a38-5946-4339-83ef-f3309bdcbf4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:19.886198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73815a38-5946-4339-83ef-f3309bdcbf4a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:19.905422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1072/2141] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:19.910689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73815a38-5946-4339-83ef-f3309bdcbf4a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:19.920952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f429c04-9b01-4ac9-a169-77951c7d4317 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:19.923349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73815a38-5946-4339-83ef-f3309bdcbf4a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:19.925996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f46c4a-7caf-457e-9576-de318f8c95e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:19.927147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f46c4a-7caf-457e-9576-de318f8c95e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:19.927533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f46c4a-7caf-457e-9576-de318f8c95e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:19.927924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f46c4a-7caf-457e-9576-de318f8c95e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:19.930165 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871a1d2-b763-48fe-a5e2-f3100a6727dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:19.930740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a1d2-b763-48fe-a5e2-f3100a6727dd admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:19.931018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a1d2-b763-48fe-a5e2-f3100a6727dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:19.931443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871a1d2-b763-48fe-a5e2-f3100a6727dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:19.932126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871a1d2-b763-48fe-a5e2-f3100a6727dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:19.934480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871a1d2-b763-48fe-a5e2-f3100a6727dd admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:19.954632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f46c4a-7caf-457e-9576-de318f8c95e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:19.957865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f46c4a-7caf-457e-9576-de318f8c95e9 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:19.978404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1070/2142] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:19.982268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1073/2143] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:19 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:19.990802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f46c4a-7caf-457e-9576-de318f8c95e9 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:20.008017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f46c4a-7caf-457e-9576-de318f8c95e9 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:20.132572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e59a8-b575-4569-b39e-20c702048683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:20.133679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e59a8-b575-4569-b39e-20c702048683 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:20.134166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e59a8-b575-4569-b39e-20c702048683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:20.134649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e59a8-b575-4569-b39e-20c702048683 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:20.135919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e59a8-b575-4569-b39e-20c702048683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:20.139810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e59a8-b575-4569-b39e-20c702048683 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:20.198809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1071/2144] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:19 2022] POST /identity/v3/users => generated 286 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:20.203757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871a1d2-b763-48fe-a5e2-f3100a6727dd admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:20.227254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871a1d2-b763-48fe-a5e2-f3100a6727dd admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:20.238164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c7c1d-cc99-48a4-8b08-797dcb6710c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:20.239659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c7c1d-cc99-48a4-8b08-797dcb6710c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:20.240448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c7c1d-cc99-48a4-8b08-797dcb6710c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:20.241154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c7c1d-cc99-48a4-8b08-797dcb6710c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:20.242979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c7c1d-cc99-48a4-8b08-797dcb6710c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:20.250064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223c7c1d-cc99-48a4-8b08-797dcb6710c2 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:20.280895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1074/2145] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:20.288698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e59a8-b575-4569-b39e-20c702048683 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:20.322389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e59a8-b575-4569-b39e-20c702048683 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:20.335302 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd00f8-d870-41e1-b82b-1337387c2a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:20.336320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd00f8-d870-41e1-b82b-1337387c2a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:20.338682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd00f8-d870-41e1-b82b-1337387c2a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:20.339186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd00f8-d870-41e1-b82b-1337387c2a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:20.343902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd00f8-d870-41e1-b82b-1337387c2a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:20.348460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd00f8-d870-41e1-b82b-1337387c2a91 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:20.384085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1072/2146] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:20.390558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c7c1d-cc99-48a4-8b08-797dcb6710c2 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:20.422242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c7c1d-cc99-48a4-8b08-797dcb6710c2 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:20.431983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897dec8-6f01-4c8f-b75a-34f62c5503f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:20.432740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897dec8-6f01-4c8f-b75a-34f62c5503f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:20.433024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897dec8-6f01-4c8f-b75a-34f62c5503f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:20.433355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897dec8-6f01-4c8f-b75a-34f62c5503f0 admin admin] PATH_INFO: `/v3/projects/9c4493f8fecb4e709bffe7d67581ae74/users/9dce1267bc1f4b34ba7c7753578d398a/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:20.456174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897dec8-6f01-4c8f-b75a-34f62c5503f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4493f8fecb4e709bffe7d67581ae74, user_id=9dce1267bc1f4b34ba7c7753578d398a, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:20.459101 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897dec8-6f01-4c8f-b75a-34f62c5503f0 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:20.479761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1075/2147] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:20 2022] PUT /identity/v3/projects/9c4493f8fecb4e709bffe7d67581ae74/users/9dce1267bc1f4b34ba7c7753578d398a/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:20.499783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20467a53-77ab-405a-8ff0-3f0700487d4a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:20.500421 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20467a53-77ab-405a-8ff0-3f0700487d4a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:20.500760 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20467a53-77ab-405a-8ff0-3f0700487d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:20.568618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-20467a53-77ab-405a-8ff0-3f0700487d4a None None] MFA Rules not processed for user `9dce1267bc1f4b34ba7c7753578d398a`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:20.570526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-20467a53-77ab-405a-8ff0-3f0700487d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:20.607187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897dec8-6f01-4c8f-b75a-34f62c5503f0 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:20.610288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b897dec8-6f01-4c8f-b75a-34f62c5503f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:20.730037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1073/2148] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:20.745469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897dec8-6f01-4c8f-b75a-34f62c5503f0 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:20.751660 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:20.752841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:20.753129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:20.753619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] PATH_INFO: `/v3/projects/aec00f289f7b40a0985bfb73f0aab63a` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:20.755152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20467a53-77ab-405a-8ff0-3f0700487d4a None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:20.755626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec00f289f7b40a0985bfb73f0aab63a)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:20.758121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:20.770138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-20467a53-77ab-405a-8ff0-3f0700487d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:20.830383 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:20.875965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:20.892104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec00f289f7b40a0985bfb73f0aab63a', 'request_id': 'req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:20.903492 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec00f289f7b40a0985bfb73f0aab63a', 'request_id': 'req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:20.910170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20467a53-77ab-405a-8ff0-3f0700487d4a None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:20.918681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec00f289f7b40a0985bfb73f0aab63a', 'request_id': 'req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:20.942013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] The token cache is being invalidate because project aec00f289f7b40a0985bfb73f0aab63a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:20.942403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:20.943280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:20.945028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1076/2149] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:20 2022] DELETE /identity/v3/projects/aec00f289f7b40a0985bfb73f0aab63a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:21.047116 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7df28e-9564-492b-8200-c945f94e0263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:21.048210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7df28e-9564-492b-8200-c945f94e0263 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:21.048658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7df28e-9564-492b-8200-c945f94e0263 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:21.049013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7df28e-9564-492b-8200-c945f94e0263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:21.053830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9d7df28e-9564-492b-8200-c945f94e0263 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:21.182492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7df28e-9564-492b-8200-c945f94e0263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:21.184366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7df28e-9564-492b-8200-c945f94e0263 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:21.197995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1074/2150] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:21.240639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:21.243026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:21.330956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11b6f7-81dc-4cd5-ae6c-5faae9a6d7b4 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:21.336794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:21.337635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:21.337926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:21.338235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] PATH_INFO: `/v3/users/9dce1267bc1f4b34ba7c7753578d398a` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:21.339640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dce1267bc1f4b34ba7c7753578d398a)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:21.341717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:21.428544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dce1267bc1f4b34ba7c7753578d398a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:21.429274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dce1267bc1f4b34ba7c7753578d398a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:21.443641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dce1267bc1f4b34ba7c7753578d398a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:21.463142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dce1267bc1f4b34ba7c7753578d398a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:21.472211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dce1267bc1f4b34ba7c7753578d398a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:21.479228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1077/2151] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:21 2022] DELETE /identity/v3/users/9dce1267bc1f4b34ba7c7753578d398a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:21.484950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06345fa-e19c-4986-b2b5-30f243334cb8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:21.485427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06345fa-e19c-4986-b2b5-30f243334cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:21.485750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06345fa-e19c-4986-b2b5-30f243334cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:21.515430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c06345fa-e19c-4986-b2b5-30f243334cb8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:21.516919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c06345fa-e19c-4986-b2b5-30f243334cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:21.581369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1075/2152] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:21.596349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:21.598385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:21.670684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305eeee-b788-4263-a7a8-2c1df06c3053 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:21.675540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f8fb7-90be-4e31-92eb-b61a18481931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:21.676121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f8fb7-90be-4e31-92eb-b61a18481931 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:21.676445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f8fb7-90be-4e31-92eb-b61a18481931 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:21.676765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f8fb7-90be-4e31-92eb-b61a18481931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:21.690507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:21.695562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f8fb7-90be-4e31-92eb-b61a18481931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:21.700574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f8fb7-90be-4e31-92eb-b61a18481931 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:21.719348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1078/2153] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:22.197040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06345fa-e19c-4986-b2b5-30f243334cb8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:22.198898 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c06345fa-e19c-4986-b2b5-30f243334cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:22.213161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06345fa-e19c-4986-b2b5-30f243334cb8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:22.220183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:22.221365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.221846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.222277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] PATH_INFO: `/v3/projects/9c4493f8fecb4e709bffe7d67581ae74` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.224716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4493f8fecb4e709bffe7d67581ae74)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:22.228275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:22.289324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:22.310130 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4493f8fecb4e709bffe7d67581ae74', 'request_id': 'req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:22.318306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4493f8fecb4e709bffe7d67581ae74', 'request_id': 'req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:22.328756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4493f8fecb4e709bffe7d67581ae74', 'request_id': 'req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:22.340455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] The token cache is being invalidate because project 9c4493f8fecb4e709bffe7d67581ae74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:22.340744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:22.341406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-57bd047f-91ed-4a5c-8f09-b2f9fa814e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:22.343064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1076/2154] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:22 2022] DELETE /identity/v3/projects/9c4493f8fecb4e709bffe7d67581ae74 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:22.349104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2de152-c0ba-4fd3-a781-d159ab2d02f0 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.349424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2de152-c0ba-4fd3-a781-d159ab2d02f0 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.349757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2de152-c0ba-4fd3-a781-d159ab2d02f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.371325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-da2de152-c0ba-4fd3-a781-d159ab2d02f0 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:22.372587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-da2de152-c0ba-4fd3-a781-d159ab2d02f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:22.443460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1079/2155] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:22.453544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0a7ab-ae61-4396-9920-26db0a9fdeea None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.453544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0a7ab-ae61-4396-9920-26db0a9fdeea None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.453756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0a7ab-ae61-4396-9920-26db0a9fdeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.476261 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2de152-c0ba-4fd3-a781-d159ab2d02f0 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:22.476603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-07b0a7ab-ae61-4396-9920-26db0a9fdeea None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:22.477870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-07b0a7ab-ae61-4396-9920-26db0a9fdeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:22.478734 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-da2de152-c0ba-4fd3-a781-d159ab2d02f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:22.486311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1077/2156] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:22.491251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0a7ab-ae61-4396-9920-26db0a9fdeea None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:22.497514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2de152-c0ba-4fd3-a781-d159ab2d02f0 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:22.505925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:22.506994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.507417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.507548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0a7ab-ae61-4396-9920-26db0a9fdeea None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:22.507810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] PATH_INFO: `/v3/users/80c525991a06473f889dd830c8d842db` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.509772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c525991a06473f889dd830c8d842db)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:22.513442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:22.515282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9978925-028d-4ca6-8e7c-107e795aaa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:22.516550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978925-028d-4ca6-8e7c-107e795aaa5a admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.517517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978925-028d-4ca6-8e7c-107e795aaa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.519301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978925-028d-4ca6-8e7c-107e795aaa5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.520006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9978925-028d-4ca6-8e7c-107e795aaa5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:22.520570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9978925-028d-4ca6-8e7c-107e795aaa5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:22.524586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9978925-028d-4ca6-8e7c-107e795aaa5a admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:22.548430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1078/2157] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:06:22.555075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9978925-028d-4ca6-8e7c-107e795aaa5a admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:22.572020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9978925-028d-4ca6-8e7c-107e795aaa5a admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:22.579546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e207c2-4090-45cd-b676-467cc3a201c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:22.580492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e207c2-4090-45cd-b676-467cc3a201c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.581008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e207c2-4090-45cd-b676-467cc3a201c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.581717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e207c2-4090-45cd-b676-467cc3a201c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.582451 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e207c2-4090-45cd-b676-467cc3a201c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:22.586848 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e207c2-4090-45cd-b676-467cc3a201c1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:22.591832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c525991a06473f889dd830c8d842db'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:22.593519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c525991a06473f889dd830c8d842db'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:22.609798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c525991a06473f889dd830c8d842db'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:22.630005 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c525991a06473f889dd830c8d842db'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:22.638806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b12eb48c-b125-4f86-b939-63c8df215bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c525991a06473f889dd830c8d842db'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:22.646609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1080/2158] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:22 2022] DELETE /identity/v3/users/80c525991a06473f889dd830c8d842db => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:22.650611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1079/2159] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:22 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:22.655030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da205b3-2106-4903-bf0b-ac2748528e45 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.655506 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da205b3-2106-4903-bf0b-ac2748528e45 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.655799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da205b3-2106-4903-bf0b-ac2748528e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.657049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e207c2-4090-45cd-b676-467cc3a201c1 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:22.662479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-07e207c2-4090-45cd-b676-467cc3a201c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:22.679925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3da205b3-2106-4903-bf0b-ac2748528e45 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:22.681254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3da205b3-2106-4903-bf0b-ac2748528e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:22.775277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e207c2-4090-45cd-b676-467cc3a201c1 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:22.783721 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1081/2160] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:22.785175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4ec59-126d-435e-9d6b-333856407a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:22.786274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4ec59-126d-435e-9d6b-333856407a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.786691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4ec59-126d-435e-9d6b-333856407a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.787161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4ec59-126d-435e-9d6b-333856407a75 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.788264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4ec59-126d-435e-9d6b-333856407a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:22.796073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4ec59-126d-435e-9d6b-333856407a75 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:22.800147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da205b3-2106-4903-bf0b-ac2748528e45 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:22.804103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3da205b3-2106-4903-bf0b-ac2748528e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:22.825469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:22.852266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1080/2161] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:22 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:22.857851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4ec59-126d-435e-9d6b-333856407a75 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:22.882757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4ec59-126d-435e-9d6b-333856407a75 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:22.892879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493ab09-ee9e-482c-806c-d7f63e40016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:22.894286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493ab09-ee9e-482c-806c-d7f63e40016c admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.895117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493ab09-ee9e-482c-806c-d7f63e40016c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.896000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493ab09-ee9e-482c-806c-d7f63e40016c admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.897881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493ab09-ee9e-482c-806c-d7f63e40016c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:22.903307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493ab09-ee9e-482c-806c-d7f63e40016c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:22.933048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1081/2162] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:22.937377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493ab09-ee9e-482c-806c-d7f63e40016c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:22.939083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da205b3-2106-4903-bf0b-ac2748528e45 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:22.943475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad76b9f-c894-4577-a718-a2527971aeba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:22.945301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad76b9f-c894-4577-a718-a2527971aeba service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.945669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad76b9f-c894-4577-a718-a2527971aeba service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.946024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad76b9f-c894-4577-a718-a2527971aeba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.950200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493ab09-ee9e-482c-806c-d7f63e40016c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:22.958987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34db71-b622-4a5b-975b-01ba3bcdb1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:22.959598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad76b9f-c894-4577-a718-a2527971aeba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:22.959955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34db71-b622-4a5b-975b-01ba3bcdb1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:22.960573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34db71-b622-4a5b-975b-01ba3bcdb1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:22.960994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34db71-b622-4a5b-975b-01ba3bcdb1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:22.961822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34db71-b622-4a5b-975b-01ba3bcdb1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:22.962174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad76b9f-c894-4577-a718-a2527971aeba service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:22.963687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34db71-b622-4a5b-975b-01ba3bcdb1a9 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:22.976575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1082/2163] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:22.977134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1082/2164] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:22.981787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34db71-b622-4a5b-975b-01ba3bcdb1a9 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:22.995268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34db71-b622-4a5b-975b-01ba3bcdb1a9 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:23.002068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660243ed-1396-41ff-9d42-1d786791c537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:23.003199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660243ed-1396-41ff-9d42-1d786791c537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:23.004200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660243ed-1396-41ff-9d42-1d786791c537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:23.004843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660243ed-1396-41ff-9d42-1d786791c537 admin admin] PATH_INFO: `/v3/projects/a0bf742a930e4c0bbc736b33cd354e2c/users/26eff167003f47188d6c922735e1653f/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:23.032100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660243ed-1396-41ff-9d42-1d786791c537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0bf742a930e4c0bbc736b33cd354e2c, user_id=26eff167003f47188d6c922735e1653f, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:23.037069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660243ed-1396-41ff-9d42-1d786791c537 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:23.074506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1083/2165] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:22 2022] PUT /identity/v3/projects/a0bf742a930e4c0bbc736b33cd354e2c/users/26eff167003f47188d6c922735e1653f/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:23.090002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e317bf-25f0-4e04-ada0-7730899cc123 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:23.090365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e317bf-25f0-4e04-ada0-7730899cc123 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:23.090597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e317bf-25f0-4e04-ada0-7730899cc123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:23.127877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-71e317bf-25f0-4e04-ada0-7730899cc123 None None] MFA Rules not processed for user `26eff167003f47188d6c922735e1653f`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:23.129297 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-71e317bf-25f0-4e04-ada0-7730899cc123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:23.219107 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1083/2166] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:23.233226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660243ed-1396-41ff-9d42-1d786791c537 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:23.235278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-660243ed-1396-41ff-9d42-1d786791c537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:23.267923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:23.325201 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660243ed-1396-41ff-9d42-1d786791c537 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:23.333223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4e175-87e0-4e94-b566-4fff1f770af0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:23.335815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4e175-87e0-4e94-b566-4fff1f770af0 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:23.336112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4e175-87e0-4e94-b566-4fff1f770af0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:23.336426 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4e175-87e0-4e94-b566-4fff1f770af0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:23.346714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4e175-87e0-4e94-b566-4fff1f770af0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:23.348345 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4e175-87e0-4e94-b566-4fff1f770af0 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:23.358570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1084/2167] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:23.593013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e317bf-25f0-4e04-ada0-7730899cc123 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:23.594648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-71e317bf-25f0-4e04-ada0-7730899cc123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:23.682036 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e317bf-25f0-4e04-ada0-7730899cc123 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:23.688111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:23.689163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:23.689621 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:23.690072 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] PATH_INFO: `/v3/projects/9d64b692cd8f4d058205c48b46abef31` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:23.691351 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d64b692cd8f4d058205c48b46abef31)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:23.693464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:23.762546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:23.768135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d64b692cd8f4d058205c48b46abef31', 'request_id': 'req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:23.778053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d64b692cd8f4d058205c48b46abef31', 'request_id': 'req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:23.790271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d64b692cd8f4d058205c48b46abef31', 'request_id': 'req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:23.814416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] The token cache is being invalidate because project 9d64b692cd8f4d058205c48b46abef31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:23.814755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:23.818058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d20ec92-c55c-44e2-92d3-b5e9a108c1c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:23.825509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1084/2168] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:23 2022] DELETE /identity/v3/projects/9d64b692cd8f4d058205c48b46abef31 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:23.832400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed456-4121-49e2-a5a0-246185094982 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:23.832400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed456-4121-49e2-a5a0-246185094982 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:23.832400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceed456-4121-49e2-a5a0-246185094982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:23.862490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-bceed456-4121-49e2-a5a0-246185094982 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:23.863818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-bceed456-4121-49e2-a5a0-246185094982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:23.943220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1085/2169] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:23.951488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b9bec-48fb-476b-8032-752207108294 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:23.951781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b9bec-48fb-476b-8032-752207108294 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:23.952059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b9bec-48fb-476b-8032-752207108294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:23.975184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9a2b9bec-48fb-476b-8032-752207108294 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:23.976505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9a2b9bec-48fb-476b-8032-752207108294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:23.985308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1085/2170] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:23.990273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceed456-4121-49e2-a5a0-246185094982 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:24.001603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceed456-4121-49e2-a5a0-246185094982 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:24.006763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce98a7-5d50-4c27-81a6-f1a9e640a969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:24.007433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce98a7-5d50-4c27-81a6-f1a9e640a969 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:24.007718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce98a7-5d50-4c27-81a6-f1a9e640a969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:24.007997 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ce98a7-5d50-4c27-81a6-f1a9e640a969 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:24.008744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce98a7-5d50-4c27-81a6-f1a9e640a969 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:24.009055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce98a7-5d50-4c27-81a6-f1a9e640a969 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:24.010813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ce98a7-5d50-4c27-81a6-f1a9e640a969 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:24.034231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1086/2171] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:24.040166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b9bec-48fb-476b-8032-752207108294 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:24.050766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b9bec-48fb-476b-8032-752207108294 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:24.057916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c81245-b605-4541-9977-11a0402acb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:24.058592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c81245-b605-4541-9977-11a0402acb75 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:24.058870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c81245-b605-4541-9977-11a0402acb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:24.059104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c81245-b605-4541-9977-11a0402acb75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:24.059846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c81245-b605-4541-9977-11a0402acb75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:24.061864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c81245-b605-4541-9977-11a0402acb75 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:24.117164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1086/2172] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:24 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:24.123833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce98a7-5d50-4c27-81a6-f1a9e640a969 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:24.147709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ce98a7-5d50-4c27-81a6-f1a9e640a969 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:24.257876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74733dca-4133-4264-8c75-b5ff5d01f537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:24.257876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74733dca-4133-4264-8c75-b5ff5d01f537 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:24.257876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74733dca-4133-4264-8c75-b5ff5d01f537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:24.257876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74733dca-4133-4264-8c75-b5ff5d01f537 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:24.258496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74733dca-4133-4264-8c75-b5ff5d01f537 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:24.258641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74733dca-4133-4264-8c75-b5ff5d01f537 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:24.293137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1087/2173] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:24 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:24.297470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c81245-b605-4541-9977-11a0402acb75 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:24.312403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c81245-b605-4541-9977-11a0402acb75 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:24.319198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5e089-2182-490f-9507-d4f6355ea729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:24.320115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5e089-2182-490f-9507-d4f6355ea729 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:24.320717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5e089-2182-490f-9507-d4f6355ea729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:24.321171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5e089-2182-490f-9507-d4f6355ea729 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:24.322171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5e089-2182-490f-9507-d4f6355ea729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:24.324504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5e089-2182-490f-9507-d4f6355ea729 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:24.339154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1087/2174] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:24.344516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74733dca-4133-4264-8c75-b5ff5d01f537 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:24.356705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74733dca-4133-4264-8c75-b5ff5d01f537 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:24.361805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a0c2b-8129-41c9-bec7-b88ba6bd9179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:24.362462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a0c2b-8129-41c9-bec7-b88ba6bd9179 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:24.362741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a0c2b-8129-41c9-bec7-b88ba6bd9179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:24.363019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a0c2b-8129-41c9-bec7-b88ba6bd9179 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:24.363830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a0c2b-8129-41c9-bec7-b88ba6bd9179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:24.365525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82a0c2b-8129-41c9-bec7-b88ba6bd9179 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:24.379630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1088/2175] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:24.385298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5e089-2182-490f-9507-d4f6355ea729 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:24.395578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5e089-2182-490f-9507-d4f6355ea729 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:24.405917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a71dcf-aea2-4844-8429-6ecc142ac9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:24.406632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a71dcf-aea2-4844-8429-6ecc142ac9ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:24.406916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a71dcf-aea2-4844-8429-6ecc142ac9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:24.407171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a71dcf-aea2-4844-8429-6ecc142ac9ed admin admin] PATH_INFO: `/v3/projects/83b59c6f908149568deed9de37a149ec/users/ab1631136ea04f5381891accca29fc6c/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:24.427324 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a71dcf-aea2-4844-8429-6ecc142ac9ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=83b59c6f908149568deed9de37a149ec, user_id=ab1631136ea04f5381891accca29fc6c, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:24.430283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a71dcf-aea2-4844-8429-6ecc142ac9ed admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:24.455227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1088/2176] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:24 2022] PUT /identity/v3/projects/83b59c6f908149568deed9de37a149ec/users/ab1631136ea04f5381891accca29fc6c/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:24.478825 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661f783-9263-4f41-97e5-8b1bfea490d6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:24.479771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661f783-9263-4f41-97e5-8b1bfea490d6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:24.480239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661f783-9263-4f41-97e5-8b1bfea490d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:24.542271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-f661f783-9263-4f41-97e5-8b1bfea490d6 None None] MFA Rules not processed for user `ab1631136ea04f5381891accca29fc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:24.546935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f661f783-9263-4f41-97e5-8b1bfea490d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:24.572908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:24.680107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1089/2177] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:24.684237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a71dcf-aea2-4844-8429-6ecc142ac9ed admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:24.798492 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a71dcf-aea2-4844-8429-6ecc142ac9ed admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:24.808806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e185a-ba3e-4d0f-8d74-eae84c2b1ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:24.809980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e185a-ba3e-4d0f-8d74-eae84c2b1ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:24.810445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e185a-ba3e-4d0f-8d74-eae84c2b1ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:24.811001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e185a-ba3e-4d0f-8d74-eae84c2b1ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:24.812271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e185a-ba3e-4d0f-8d74-eae84c2b1ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:24.815363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0e185a-ba3e-4d0f-8d74-eae84c2b1ef4 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:24.866444 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1089/2178] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:24 2022] POST /identity/v3/projects => generated 377 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:24.873526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661f783-9263-4f41-97e5-8b1bfea490d6 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:24.899783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661f783-9263-4f41-97e5-8b1bfea490d6 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:25.022942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9b34e-9f61-4812-aa8b-f94da22eff64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:25.024280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9b34e-9f61-4812-aa8b-f94da22eff64 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.024597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9b34e-9f61-4812-aa8b-f94da22eff64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.025140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9b34e-9f61-4812-aa8b-f94da22eff64 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.025976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9b34e-9f61-4812-aa8b-f94da22eff64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:25.028238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe9b34e-9f61-4812-aa8b-f94da22eff64 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:25.061311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1090/2179] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:24 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:25.065897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e185a-ba3e-4d0f-8d74-eae84c2b1ef4 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:25.084402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0e185a-ba3e-4d0f-8d74-eae84c2b1ef4 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:25.095834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f4a2-88a2-46e1-acd6-886476208c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:25.096736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f4a2-88a2-46e1-acd6-886476208c31 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.097199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f4a2-88a2-46e1-acd6-886476208c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.097957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20f4a2-88a2-46e1-acd6-886476208c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.100048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20f4a2-88a2-46e1-acd6-886476208c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:25.103017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20f4a2-88a2-46e1-acd6-886476208c31 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:25.128505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1090/2180] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:25.135155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9b34e-9f61-4812-aa8b-f94da22eff64 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:25.160742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9b34e-9f61-4812-aa8b-f94da22eff64 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:25.172269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df66d0-6a6e-4c57-a08d-fdac472f2bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:25.173784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df66d0-6a6e-4c57-a08d-fdac472f2bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.174421 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df66d0-6a6e-4c57-a08d-fdac472f2bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.176683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df66d0-6a6e-4c57-a08d-fdac472f2bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.178249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df66d0-6a6e-4c57-a08d-fdac472f2bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:25.181312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df66d0-6a6e-4c57-a08d-fdac472f2bc2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:25.209628 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1091/2181] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:25.214216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f4a2-88a2-46e1-acd6-886476208c31 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:25.237972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20f4a2-88a2-46e1-acd6-886476208c31 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:25.251823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c6c1f-6f35-4b93-8abf-08154c5779df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:25.252742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c6c1f-6f35-4b93-8abf-08154c5779df admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.253059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c6c1f-6f35-4b93-8abf-08154c5779df admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.253363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c6c1f-6f35-4b93-8abf-08154c5779df admin admin] PATH_INFO: `/v3/projects/86a46298236c4b60b83fb8d6b37127ee/users/3b918603f937441e82061b6cc4c58121/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.286351 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c6c1f-6f35-4b93-8abf-08154c5779df admin admin] RBAC: Authorizing `identity:create_grant(project_id=86a46298236c4b60b83fb8d6b37127ee, user_id=3b918603f937441e82061b6cc4c58121, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:25.289402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c6c1f-6f35-4b93-8abf-08154c5779df admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:25.314347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1091/2182] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:25 2022] PUT /identity/v3/projects/86a46298236c4b60b83fb8d6b37127ee/users/3b918603f937441e82061b6cc4c58121/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:25.328117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7497a75-6e2d-4461-8074-43b1d319facf None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.329371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7497a75-6e2d-4461-8074-43b1d319facf None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.330012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7497a75-6e2d-4461-8074-43b1d319facf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.378569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a7497a75-6e2d-4461-8074-43b1d319facf None None] MFA Rules not processed for user `3b918603f937441e82061b6cc4c58121`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:25.379115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a7497a75-6e2d-4461-8074-43b1d319facf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:25.462315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1092/2183] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:25.467067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c6c1f-6f35-4b93-8abf-08154c5779df admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:25.489187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:25.549705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c6c1f-6f35-4b93-8abf-08154c5779df admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:25.554971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e8cbf-5c00-4914-a3ea-a52304e97ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:25.555627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e8cbf-5c00-4914-a3ea-a52304e97ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.555905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e8cbf-5c00-4914-a3ea-a52304e97ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.556175 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e8cbf-5c00-4914-a3ea-a52304e97ee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.556921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e8cbf-5c00-4914-a3ea-a52304e97ee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:25.558889 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e8cbf-5c00-4914-a3ea-a52304e97ee5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:25.594144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1092/2184] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:25 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:25.595368 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7497a75-6e2d-4461-8074-43b1d319facf None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:25.606883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7497a75-6e2d-4461-8074-43b1d319facf None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:25.630716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:25.674258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cfdb8-609d-4c5e-99fa-8ddb8938cca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:25.674823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cfdb8-609d-4c5e-99fa-8ddb8938cca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.675100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cfdb8-609d-4c5e-99fa-8ddb8938cca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.675376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cfdb8-609d-4c5e-99fa-8ddb8938cca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.676163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cfdb8-609d-4c5e-99fa-8ddb8938cca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:25.678239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cfdb8-609d-4c5e-99fa-8ddb8938cca6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:25.702206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1093/2185] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:25 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:25.706631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e8cbf-5c00-4914-a3ea-a52304e97ee5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:25.718996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e8cbf-5c00-4914-a3ea-a52304e97ee5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:25.728327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982f76e-3f2c-48f6-b254-5abfe892e951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:25.729014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982f76e-3f2c-48f6-b254-5abfe892e951 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.729295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982f76e-3f2c-48f6-b254-5abfe892e951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.729569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982f76e-3f2c-48f6-b254-5abfe892e951 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.730330 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982f76e-3f2c-48f6-b254-5abfe892e951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:25.732017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982f76e-3f2c-48f6-b254-5abfe892e951 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:25.745784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1093/2186] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:25.751416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cfdb8-609d-4c5e-99fa-8ddb8938cca6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:25.762441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cfdb8-609d-4c5e-99fa-8ddb8938cca6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:25.768461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f0ee9-95fb-495c-a1f0-4b487ac3eb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:25.769175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f0ee9-95fb-495c-a1f0-4b487ac3eb45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.769459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f0ee9-95fb-495c-a1f0-4b487ac3eb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.769826 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f0ee9-95fb-495c-a1f0-4b487ac3eb45 admin admin] PATH_INFO: `/v3/projects/3ddcd13794b54a39ae3fe1cb6c471a60/users/5f0e83344c4d4e0193bb32a86a5bf781/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.791179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f0ee9-95fb-495c-a1f0-4b487ac3eb45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ddcd13794b54a39ae3fe1cb6c471a60, user_id=5f0e83344c4d4e0193bb32a86a5bf781, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:25.794996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f0ee9-95fb-495c-a1f0-4b487ac3eb45 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:25.811505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1094/2187] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:25 2022] PUT /identity/v3/projects/3ddcd13794b54a39ae3fe1cb6c471a60/users/5f0e83344c4d4e0193bb32a86a5bf781/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:25.815892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982f76e-3f2c-48f6-b254-5abfe892e951 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:25.827748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982f76e-3f2c-48f6-b254-5abfe892e951 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:25.944496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc172b-686c-411d-aa50-872aa12329cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:25.945103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc172b-686c-411d-aa50-872aa12329cd admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.945388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc172b-686c-411d-aa50-872aa12329cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.945659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc172b-686c-411d-aa50-872aa12329cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.946550 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc172b-686c-411d-aa50-872aa12329cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:25.948430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc172b-686c-411d-aa50-872aa12329cd admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:25.961705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1094/2188] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:25.966152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f0ee9-95fb-495c-a1f0-4b487ac3eb45 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:25.977737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f0ee9-95fb-495c-a1f0-4b487ac3eb45 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:25.983416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d968a-ec1b-447d-8ce6-91e76554f82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:25.984209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d968a-ec1b-447d-8ce6-91e76554f82c admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:25.984783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d968a-ec1b-447d-8ce6-91e76554f82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:25.985432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d968a-ec1b-447d-8ce6-91e76554f82c admin admin] PATH_INFO: `/v3/projects/3ddcd13794b54a39ae3fe1cb6c471a60/users/5f0e83344c4d4e0193bb32a86a5bf781/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:25.987888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d968a-ec1b-447d-8ce6-91e76554f82c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ddcd13794b54a39ae3fe1cb6c471a60, user_id=5f0e83344c4d4e0193bb32a86a5bf781, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:25.991049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d968a-ec1b-447d-8ce6-91e76554f82c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:26.032074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1095/2189] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:25 2022] PUT /identity/v3/projects/3ddcd13794b54a39ae3fe1cb6c471a60/users/5f0e83344c4d4e0193bb32a86a5bf781/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:26.036618 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541e8e7-6eb2-4510-9797-d4619361f7b1 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:26.038091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541e8e7-6eb2-4510-9797-d4619361f7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:26.038874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541e8e7-6eb2-4510-9797-d4619361f7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:26.084917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3541e8e7-6eb2-4510-9797-d4619361f7b1 None None] MFA Rules not processed for user `5f0e83344c4d4e0193bb32a86a5bf781`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:26.086385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3541e8e7-6eb2-4510-9797-d4619361f7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:26.214112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1095/2190] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:26.225892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d968a-ec1b-447d-8ce6-91e76554f82c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:26.227679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b57d968a-ec1b-447d-8ce6-91e76554f82c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:26.302551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d968a-ec1b-447d-8ce6-91e76554f82c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:26.306425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9b033-e977-4cb9-be27-dfc054bed8fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:26.307088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9b033-e977-4cb9-be27-dfc054bed8fb service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:26.307381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9b033-e977-4cb9-be27-dfc054bed8fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:26.307628 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc9b033-e977-4cb9-be27-dfc054bed8fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:26.454826 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9b033-e977-4cb9-be27-dfc054bed8fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:26.456497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc9b033-e977-4cb9-be27-dfc054bed8fb service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:26.471047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1096/2191] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:27.726273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541e8e7-6eb2-4510-9797-d4619361f7b1 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:27.727934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3541e8e7-6eb2-4510-9797-d4619361f7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:27.752616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:27.834981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541e8e7-6eb2-4510-9797-d4619361f7b1 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:27.853995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:27.854779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:27.855124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:27.855465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] PATH_INFO: `/v3/users/b76e6a76bddb48a98d64e7b2dfcc9eff` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:27.857060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b76e6a76bddb48a98d64e7b2dfcc9eff)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:27.859455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:27.933401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b76e6a76bddb48a98d64e7b2dfcc9eff'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:27.934111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b76e6a76bddb48a98d64e7b2dfcc9eff'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:27.948608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b76e6a76bddb48a98d64e7b2dfcc9eff'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:27.967641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b76e6a76bddb48a98d64e7b2dfcc9eff'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:27.976783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b76e6a76bddb48a98d64e7b2dfcc9eff'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:27.985086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1096/2192] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:27 2022] DELETE /identity/v3/users/b76e6a76bddb48a98d64e7b2dfcc9eff => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:28.241196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9b033-e977-4cb9-be27-dfc054bed8fb service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:28.263303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc9b033-e977-4cb9-be27-dfc054bed8fb service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:28.303662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:28.351540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:28.353208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:28.354055 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:28.354648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] PATH_INFO: `/v3/projects/d0f6b755d6624595a58cd76cbf296e00` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:28.362007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0f6b755d6624595a58cd76cbf296e00)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:28.372706 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:28.450108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0f6b755d6624595a58cd76cbf296e00', 'request_id': 'req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.460653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0f6b755d6624595a58cd76cbf296e00', 'request_id': 'req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.474354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0f6b755d6624595a58cd76cbf296e00', 'request_id': 'req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.499934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] The token cache is being invalidate because project d0f6b755d6624595a58cd76cbf296e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:28.500215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.501029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.502975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1097/2193] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:28 2022] DELETE /identity/v3/projects/d0f6b755d6624595a58cd76cbf296e00 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:28.507324 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:28.509344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:28.525433 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38b8dd-691d-4b23-8f90-c0327476964a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:28.600968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:28.601738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:28.602144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:28.602485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] PATH_INFO: `/v3/users/1b0e96724e824414bfce15498d7a1659` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:28.603821 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b0e96724e824414bfce15498d7a1659)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:28.606030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:28.670449 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:28.689524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b0e96724e824414bfce15498d7a1659'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.690819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b0e96724e824414bfce15498d7a1659'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.701993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b0e96724e824414bfce15498d7a1659'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.741554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b0e96724e824414bfce15498d7a1659'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.752035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b0e96724e824414bfce15498d7a1659'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.763148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1097/2194] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:28 2022] DELETE /identity/v3/users/1b0e96724e824414bfce15498d7a1659 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:28.802343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:28.804119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:28.823935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d7c90-9ca5-42e7-bb9f-d0f926aa56d6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:28.906030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:28.906706 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:28.906988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:28.907263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] PATH_INFO: `/v3/users/9bc6d25504f1428a84ee2cd93657b9d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:28.908533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bc6d25504f1428a84ee2cd93657b9d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:28.910627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:28.978546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bc6d25504f1428a84ee2cd93657b9d1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:28.991980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:28.993805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bc6d25504f1428a84ee2cd93657b9d1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.003488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bc6d25504f1428a84ee2cd93657b9d1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.005072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bc6d25504f1428a84ee2cd93657b9d1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.006045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bc6d25504f1428a84ee2cd93657b9d1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.022270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1098/2195] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:28 2022] DELETE /identity/v3/users/9bc6d25504f1428a84ee2cd93657b9d1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:29.044695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:29.046414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:29.060529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8145c1-9731-4794-88c0-c2e2c2dcf20f admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:29.179151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:29.179951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.180336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.180615 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] PATH_INFO: `/v3/projects/950f0067d6b644249ab9e593d3617344` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.182014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=950f0067d6b644249ab9e593d3617344)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:29.184058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:29.278673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950f0067d6b644249ab9e593d3617344', 'request_id': 'req-00be43e2-2a35-4856-9222-e144140c02c1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.291632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950f0067d6b644249ab9e593d3617344', 'request_id': 'req-00be43e2-2a35-4856-9222-e144140c02c1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.305194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950f0067d6b644249ab9e593d3617344', 'request_id': 'req-00be43e2-2a35-4856-9222-e144140c02c1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.306456 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:29.307946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:29.323290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] The token cache is being invalidate because project 950f0067d6b644249ab9e593d3617344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:29.323578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.324343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-00be43e2-2a35-4856-9222-e144140c02c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.325843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1098/2196] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:29 2022] DELETE /identity/v3/projects/950f0067d6b644249ab9e593d3617344 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:29.326146 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a6820-80b3-41ba-848c-c4e2e45d3014 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:29.334713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac851ca-fbfd-493d-b87f-a301f6b15b74 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.335000 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac851ca-fbfd-493d-b87f-a301f6b15b74 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.335280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac851ca-fbfd-493d-b87f-a301f6b15b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.362539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4ac851ca-fbfd-493d-b87f-a301f6b15b74 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:29.363891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4ac851ca-fbfd-493d-b87f-a301f6b15b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:29.404869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:29.405757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.406212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.406646 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] PATH_INFO: `/v3/projects/9126d4b7b81640b19ba5f62236e19444` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.408647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:29.408816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9126d4b7b81640b19ba5f62236e19444)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:29.411428 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:29.451121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1099/2197] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:29.451997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac851ca-fbfd-493d-b87f-a301f6b15b74 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:29.453332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4ac851ca-fbfd-493d-b87f-a301f6b15b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:29.482324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9126d4b7b81640b19ba5f62236e19444', 'request_id': 'req-fd5de38d-03f8-40f6-8c99-017bf24efabb'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.490631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9126d4b7b81640b19ba5f62236e19444', 'request_id': 'req-fd5de38d-03f8-40f6-8c99-017bf24efabb'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.500249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9126d4b7b81640b19ba5f62236e19444', 'request_id': 'req-fd5de38d-03f8-40f6-8c99-017bf24efabb'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.513977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] The token cache is being invalidate because project 9126d4b7b81640b19ba5f62236e19444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:29.514319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.514947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-fd5de38d-03f8-40f6-8c99-017bf24efabb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:29.516697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1099/2198] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:29 2022] DELETE /identity/v3/projects/9126d4b7b81640b19ba5f62236e19444 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:29.518031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a1a28-5b6c-4837-b132-d83cabf9acae None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.518323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a1a28-5b6c-4837-b132-d83cabf9acae None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.518532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a1a28-5b6c-4837-b132-d83cabf9acae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.544170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-2c2a1a28-5b6c-4837-b132-d83cabf9acae None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:29.545453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2c2a1a28-5b6c-4837-b132-d83cabf9acae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:29.547982 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac851ca-fbfd-493d-b87f-a301f6b15b74 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:29.624393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1100/2199] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:29.624393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dc578-5f41-4fe8-a7ad-d0540d447eb8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.624393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dc578-5f41-4fe8-a7ad-d0540d447eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.624393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03dc578-5f41-4fe8-a7ad-d0540d447eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.627949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9195774-ae67-45d5-a8b2-142854e1c1a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:29.629140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9195774-ae67-45d5-a8b2-142854e1c1a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.629635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9195774-ae67-45d5-a8b2-142854e1c1a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.630129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9195774-ae67-45d5-a8b2-142854e1c1a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.633214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e9195774-ae67-45d5-a8b2-142854e1c1a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:29.644846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c03dc578-5f41-4fe8-a7ad-d0540d447eb8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:29.646108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c03dc578-5f41-4fe8-a7ad-d0540d447eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:29.664364 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1101/2200] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:29.664364 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dc578-5f41-4fe8-a7ad-d0540d447eb8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:29.670072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:29.674962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03dc578-5f41-4fe8-a7ad-d0540d447eb8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:29.681736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f674f9-826c-4cb6-9e67-615f2bbf1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:29.682486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f674f9-826c-4cb6-9e67-615f2bbf1f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.682739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f674f9-826c-4cb6-9e67-615f2bbf1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.683018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f674f9-826c-4cb6-9e67-615f2bbf1f04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.683882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f674f9-826c-4cb6-9e67-615f2bbf1f04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:29.684195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f674f9-826c-4cb6-9e67-615f2bbf1f04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:29.686207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f674f9-826c-4cb6-9e67-615f2bbf1f04 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:29.704621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1102/2201] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:29.707042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e44ef-a3f0-4784-8804-025ec94e4ab8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.707542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e44ef-a3f0-4784-8804-025ec94e4ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.708012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e44ef-a3f0-4784-8804-025ec94e4ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.723096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9195774-ae67-45d5-a8b2-142854e1c1a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:29.724117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9195774-ae67-45d5-a8b2-142854e1c1a7 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:29.730327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3f2e44ef-a3f0-4784-8804-025ec94e4ab8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:29.731576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3f2e44ef-a3f0-4784-8804-025ec94e4ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:29.735789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1100/2202] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:29.736956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9195774-ae67-45d5-a8b2-142854e1c1a7 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:29.748861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1103/2203] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:29.750275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9195774-ae67-45d5-a8b2-142854e1c1a7 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:29.754017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e44ef-a3f0-4784-8804-025ec94e4ab8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:29.756808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd7488-d343-4356-a3fa-3589097e60ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:29.757464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd7488-d343-4356-a3fa-3589097e60ab admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.757738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd7488-d343-4356-a3fa-3589097e60ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.758009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd7488-d343-4356-a3fa-3589097e60ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.759143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd7488-d343-4356-a3fa-3589097e60ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:29.761850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd7488-d343-4356-a3fa-3589097e60ab admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:29.763867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e44ef-a3f0-4784-8804-025ec94e4ab8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:29.768943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb9518-bb02-4cf1-94c7-cc7ec5bb04f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:29.769857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb9518-bb02-4cf1-94c7-cc7ec5bb04f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.770043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb9518-bb02-4cf1-94c7-cc7ec5bb04f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.770326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb9518-bb02-4cf1-94c7-cc7ec5bb04f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.772844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb9518-bb02-4cf1-94c7-cc7ec5bb04f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:29.772844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb9518-bb02-4cf1-94c7-cc7ec5bb04f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:29.774472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb9518-bb02-4cf1-94c7-cc7ec5bb04f7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:29.796163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1104/2204] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:29.802892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb9518-bb02-4cf1-94c7-cc7ec5bb04f7 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:29.814183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1101/2205] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:29 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:29.818216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb9518-bb02-4cf1-94c7-cc7ec5bb04f7 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:29.820971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd7488-d343-4356-a3fa-3589097e60ab admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:29.879457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:29.917671 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba95372-9596-423d-a732-87057a3b59ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:29.918811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba95372-9596-423d-a732-87057a3b59ae admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.919008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba95372-9596-423d-a732-87057a3b59ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.919280 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba95372-9596-423d-a732-87057a3b59ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.920057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba95372-9596-423d-a732-87057a3b59ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:29.922154 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba95372-9596-423d-a732-87057a3b59ae admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:29.940958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd7488-d343-4356-a3fa-3589097e60ab admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:29.948757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8007864-8440-48fc-a8d6-0af77276344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:29.949411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8007864-8440-48fc-a8d6-0af77276344c admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:29.949697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8007864-8440-48fc-a8d6-0af77276344c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:29.949952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8007864-8440-48fc-a8d6-0af77276344c admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:29.950625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8007864-8440-48fc-a8d6-0af77276344c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:29.952669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8007864-8440-48fc-a8d6-0af77276344c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:29.955647 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1105/2206] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:29 2022] POST /identity/v3/projects => generated 363 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:29.960883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba95372-9596-423d-a732-87057a3b59ae admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:29.980428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1102/2207] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:29 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:29.986666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8007864-8440-48fc-a8d6-0af77276344c admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:29.999909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8007864-8440-48fc-a8d6-0af77276344c admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.106040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e0164-474b-40fe-8638-dd77331ede77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.107222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e0164-474b-40fe-8638-dd77331ede77 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.107832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e0164-474b-40fe-8638-dd77331ede77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.108434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e0164-474b-40fe-8638-dd77331ede77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.109283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e0164-474b-40fe-8638-dd77331ede77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.111204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e0164-474b-40fe-8638-dd77331ede77 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.121583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba95372-9596-423d-a732-87057a3b59ae admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.133457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1103/2208] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:30.133567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4f60d-70d9-4578-9f43-1114572fceda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.133567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4f60d-70d9-4578-9f43-1114572fceda admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.133567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4f60d-70d9-4578-9f43-1114572fceda admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.133567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4f60d-70d9-4578-9f43-1114572fceda admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.134166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4f60d-70d9-4578-9f43-1114572fceda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.136636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e0164-474b-40fe-8638-dd77331ede77 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.138147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4f60d-70d9-4578-9f43-1114572fceda admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.149498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e0164-474b-40fe-8638-dd77331ede77 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.158926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbe670-f183-4440-8ca5-9326a1645a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.160786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbe670-f183-4440-8ca5-9326a1645a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.161706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbe670-f183-4440-8ca5-9326a1645a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.162014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cbe670-f183-4440-8ca5-9326a1645a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.162983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbe670-f183-4440-8ca5-9326a1645a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.164778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cbe670-f183-4440-8ca5-9326a1645a39 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.176945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1104/2209] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:30.182832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbe670-f183-4440-8ca5-9326a1645a39 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.187314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1106/2210] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:29 2022] POST /identity/v3/users => generated 282 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:30.194233 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4f60d-70d9-4578-9f43-1114572fceda admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.203562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cbe670-f183-4440-8ca5-9326a1645a39 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.207571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4f60d-70d9-4578-9f43-1114572fceda admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.212596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948526d-9c55-4112-bdde-d24fbe6ff053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.213199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842052d1-7237-4800-a3b6-1448235f32e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.213399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948526d-9c55-4112-bdde-d24fbe6ff053 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.213543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948526d-9c55-4112-bdde-d24fbe6ff053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.213800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948526d-9c55-4112-bdde-d24fbe6ff053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.214544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948526d-9c55-4112-bdde-d24fbe6ff053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.214710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842052d1-7237-4800-a3b6-1448235f32e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.215197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842052d1-7237-4800-a3b6-1448235f32e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.215786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842052d1-7237-4800-a3b6-1448235f32e2 admin admin] PATH_INFO: `/v3/projects/37eab4b79bda465186e12a8d8ddf58e2/users/2e8e0e68c64b488b8c08cc743385e873/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.216175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6948526d-9c55-4112-bdde-d24fbe6ff053 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.229444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1107/2211] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:30.233570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948526d-9c55-4112-bdde-d24fbe6ff053 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.244875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948526d-9c55-4112-bdde-d24fbe6ff053 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.245934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842052d1-7237-4800-a3b6-1448235f32e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37eab4b79bda465186e12a8d8ddf58e2, user_id=2e8e0e68c64b488b8c08cc743385e873, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.250022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c94c3-bbfd-4a81-a9a9-a7e925ec1630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.250669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c94c3-bbfd-4a81-a9a9-a7e925ec1630 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.251089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c94c3-bbfd-4a81-a9a9-a7e925ec1630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.251343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842052d1-7237-4800-a3b6-1448235f32e2 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.251521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c94c3-bbfd-4a81-a9a9-a7e925ec1630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.252246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c94c3-bbfd-4a81-a9a9-a7e925ec1630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.253967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c94c3-bbfd-4a81-a9a9-a7e925ec1630 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.267406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1108/2212] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:30.269137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1105/2213] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:30 2022] PUT /identity/v3/projects/37eab4b79bda465186e12a8d8ddf58e2/users/2e8e0e68c64b488b8c08cc743385e873/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:30.272958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c94c3-bbfd-4a81-a9a9-a7e925ec1630 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.284750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c94c3-bbfd-4a81-a9a9-a7e925ec1630 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.287735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fca6a6-d9e8-4098-ab6e-81525683c073 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.288014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fca6a6-d9e8-4098-ab6e-81525683c073 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.288283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fca6a6-d9e8-4098-ab6e-81525683c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.317682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c0fca6a6-d9e8-4098-ab6e-81525683c073 None None] MFA Rules not processed for user `2e8e0e68c64b488b8c08cc743385e873`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:30.319028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c0fca6a6-d9e8-4098-ab6e-81525683c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:30.319187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:30.361334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7449c2-0fd9-4db2-aff7-2007eadf677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.362058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7449c2-0fd9-4db2-aff7-2007eadf677e admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.362381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7449c2-0fd9-4db2-aff7-2007eadf677e admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.362692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7449c2-0fd9-4db2-aff7-2007eadf677e admin admin] PATH_INFO: `/v3/projects/bbfb9b6e55a149cab7cddf262353e0cc/users/f7dff0370f794a93bfc61948e1845ec1/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.385007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7449c2-0fd9-4db2-aff7-2007eadf677e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbfb9b6e55a149cab7cddf262353e0cc, user_id=f7dff0370f794a93bfc61948e1845ec1, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.387659 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7449c2-0fd9-4db2-aff7-2007eadf677e admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.411157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1106/2214] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:30.415101 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1109/2215] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:30 2022] PUT /identity/v3/projects/bbfb9b6e55a149cab7cddf262353e0cc/users/f7dff0370f794a93bfc61948e1845ec1/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:30.416065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fca6a6-d9e8-4098-ab6e-81525683c073 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.428774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86d99-3c12-4b79-8c01-03308576d231 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.429148 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fca6a6-d9e8-4098-ab6e-81525683c073 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.429259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86d99-3c12-4b79-8c01-03308576d231 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.429718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd86d99-3c12-4b79-8c01-03308576d231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.441070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:30.466201 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-fbd86d99-3c12-4b79-8c01-03308576d231 None None] MFA Rules not processed for user `f7dff0370f794a93bfc61948e1845ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:30.467477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fbd86d99-3c12-4b79-8c01-03308576d231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:30.520084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e61218-bdfe-4210-bb2e-83a1c4aa44a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.521049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e61218-bdfe-4210-bb2e-83a1c4aa44a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.521291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e61218-bdfe-4210-bb2e-83a1c4aa44a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.521711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e61218-bdfe-4210-bb2e-83a1c4aa44a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.522964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e61218-bdfe-4210-bb2e-83a1c4aa44a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.525686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e61218-bdfe-4210-bb2e-83a1c4aa44a7 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.552354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1110/2216] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:30 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:30.554360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1107/2217] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:30 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:30.557187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd86d99-3c12-4b79-8c01-03308576d231 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.558490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e61218-bdfe-4210-bb2e-83a1c4aa44a7 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.570223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd86d99-3c12-4b79-8c01-03308576d231 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.624079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e61218-bdfe-4210-bb2e-83a1c4aa44a7 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.630196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1719dc-064b-4ea8-96c8-4ab313187d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.630828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1719dc-064b-4ea8-96c8-4ab313187d5e admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.631105 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1719dc-064b-4ea8-96c8-4ab313187d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.631377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1719dc-064b-4ea8-96c8-4ab313187d5e admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.632048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1719dc-064b-4ea8-96c8-4ab313187d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.634030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1719dc-064b-4ea8-96c8-4ab313187d5e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.635194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b4dfa-0ac0-4ef5-91bf-3375bee95657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.635638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b4dfa-0ac0-4ef5-91bf-3375bee95657 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.635916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b4dfa-0ac0-4ef5-91bf-3375bee95657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.636190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b4dfa-0ac0-4ef5-91bf-3375bee95657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.636893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b4dfa-0ac0-4ef5-91bf-3375bee95657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.638809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b4dfa-0ac0-4ef5-91bf-3375bee95657 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.657891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1108/2218] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:30 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:30.663083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1719dc-064b-4ea8-96c8-4ab313187d5e admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.668249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1111/2219] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:30 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:30.672384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b4dfa-0ac0-4ef5-91bf-3375bee95657 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.674229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1719dc-064b-4ea8-96c8-4ab313187d5e admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.700337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:30.738964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cf6db-0b54-4651-83b8-bf22fed18cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.739629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cf6db-0b54-4651-83b8-bf22fed18cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.739905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cf6db-0b54-4651-83b8-bf22fed18cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.740174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5cf6db-0b54-4651-83b8-bf22fed18cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.740905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cf6db-0b54-4651-83b8-bf22fed18cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.742580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5cf6db-0b54-4651-83b8-bf22fed18cd9 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.743563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b4dfa-0ac0-4ef5-91bf-3375bee95657 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.748458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cf6fb-9ddd-4c2e-81ee-61461a95820b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.749093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cf6fb-9ddd-4c2e-81ee-61461a95820b admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.749366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cf6fb-9ddd-4c2e-81ee-61461a95820b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.749633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cf6fb-9ddd-4c2e-81ee-61461a95820b admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.750687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cf6fb-9ddd-4c2e-81ee-61461a95820b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.752249 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cf6fb-9ddd-4c2e-81ee-61461a95820b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.753920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1109/2220] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:30.758428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cf6db-0b54-4651-83b8-bf22fed18cd9 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.770267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5cf6db-0b54-4651-83b8-bf22fed18cd9 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.775498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14486b7f-379e-40e9-8caf-c66611871988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.775846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1112/2221] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:30 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:30.776328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14486b7f-379e-40e9-8caf-c66611871988 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.776616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14486b7f-379e-40e9-8caf-c66611871988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.776875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14486b7f-379e-40e9-8caf-c66611871988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.777579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14486b7f-379e-40e9-8caf-c66611871988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.779537 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14486b7f-379e-40e9-8caf-c66611871988 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.779791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cf6fb-9ddd-4c2e-81ee-61461a95820b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.789691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cf6fb-9ddd-4c2e-81ee-61461a95820b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.791637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1110/2222] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:30.794257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e380-6643-4049-a199-6bf463c6c077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.795091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e380-6643-4049-a199-6bf463c6c077 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.795284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e380-6643-4049-a199-6bf463c6c077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.795558 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722e380-6643-4049-a199-6bf463c6c077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.796265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722e380-6643-4049-a199-6bf463c6c077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.797424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14486b7f-379e-40e9-8caf-c66611871988 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.801812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722e380-6643-4049-a199-6bf463c6c077 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.809398 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14486b7f-379e-40e9-8caf-c66611871988 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.818154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e90dc7-f459-4eab-adab-39bcb8351318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.819332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e90dc7-f459-4eab-adab-39bcb8351318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.819774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e90dc7-f459-4eab-adab-39bcb8351318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.820158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e90dc7-f459-4eab-adab-39bcb8351318 admin admin] PATH_INFO: `/v3/projects/d8880914a74748e79395722a9be4f5b4/users/cceeda2b1d5d45339ccb2fb0f13cbbf7/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.824885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1113/2223] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:30.830434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e380-6643-4049-a199-6bf463c6c077 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:30.837971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:30.842117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e90dc7-f459-4eab-adab-39bcb8351318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8880914a74748e79395722a9be4f5b4, user_id=cceeda2b1d5d45339ccb2fb0f13cbbf7, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.845787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e90dc7-f459-4eab-adab-39bcb8351318 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.850659 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722e380-6643-4049-a199-6bf463c6c077 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:30.869588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ec619-e9cb-4fb3-89bd-55a0138ac1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:30.870679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ec619-e9cb-4fb3-89bd-55a0138ac1b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.871043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ec619-e9cb-4fb3-89bd-55a0138ac1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.871366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ec619-e9cb-4fb3-89bd-55a0138ac1b6 admin admin] PATH_INFO: `/v3/projects/9f846197274944f7b2f7c46a9a78b8d7/users/1cdd046dfdb041158ed33e5902246921/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.871669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1111/2224] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:30 2022] PUT /identity/v3/projects/d8880914a74748e79395722a9be4f5b4/users/cceeda2b1d5d45339ccb2fb0f13cbbf7/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:30.884965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de771980-7533-44df-9c7a-1e23d09c19e7 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:30.885482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de771980-7533-44df-9c7a-1e23d09c19e7 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:30.885482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de771980-7533-44df-9c7a-1e23d09c19e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:30.904572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ec619-e9cb-4fb3-89bd-55a0138ac1b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f846197274944f7b2f7c46a9a78b8d7, user_id=1cdd046dfdb041158ed33e5902246921, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:30.907367 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ec619-e9cb-4fb3-89bd-55a0138ac1b6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:30.921270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-de771980-7533-44df-9c7a-1e23d09c19e7 None None] MFA Rules not processed for user `cceeda2b1d5d45339ccb2fb0f13cbbf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:30.922507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-de771980-7533-44df-9c7a-1e23d09c19e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:30.933177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1114/2225] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:30 2022] PUT /identity/v3/projects/9f846197274944f7b2f7c46a9a78b8d7/users/1cdd046dfdb041158ed33e5902246921/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:30.938879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ec619-e9cb-4fb3-89bd-55a0138ac1b6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.029947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1112/2226] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:31.035366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de771980-7533-44df-9c7a-1e23d09c19e7 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.047086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ec619-e9cb-4fb3-89bd-55a0138ac1b6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.048267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de771980-7533-44df-9c7a-1e23d09c19e7 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.052242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bc6ca-67fb-41af-a0ab-c4a9521a5686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.052919 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bc6ca-67fb-41af-a0ab-c4a9521a5686 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.053203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bc6ca-67fb-41af-a0ab-c4a9521a5686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.053476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bc6ca-67fb-41af-a0ab-c4a9521a5686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.054184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bc6ca-67fb-41af-a0ab-c4a9521a5686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.054467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963fde8-3425-47f7-b113-e9580ee1b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.055116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963fde8-3425-47f7-b113-e9580ee1b51c admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.055426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963fde8-3425-47f7-b113-e9580ee1b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.055707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963fde8-3425-47f7-b113-e9580ee1b51c admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.055973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bc6ca-67fb-41af-a0ab-c4a9521a5686 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.056457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963fde8-3425-47f7-b113-e9580ee1b51c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.058381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963fde8-3425-47f7-b113-e9580ee1b51c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.069492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1115/2227] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:31.073736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bc6ca-67fb-41af-a0ab-c4a9521a5686 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.091431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bc6ca-67fb-41af-a0ab-c4a9521a5686 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.104109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6d7fb-c92d-434b-a82f-db98a5d98ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.105286 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6d7fb-c92d-434b-a82f-db98a5d98ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.105767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6d7fb-c92d-434b-a82f-db98a5d98ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.106239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6d7fb-c92d-434b-a82f-db98a5d98ec7 admin admin] PATH_INFO: `/v3/projects/9f846197274944f7b2f7c46a9a78b8d7/users/1cdd046dfdb041158ed33e5902246921/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.108745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1113/2228] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:31 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:31.110401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6d7fb-c92d-434b-a82f-db98a5d98ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f846197274944f7b2f7c46a9a78b8d7, user_id=1cdd046dfdb041158ed33e5902246921, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.110795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963fde8-3425-47f7-b113-e9580ee1b51c admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.114258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6d7fb-c92d-434b-a82f-db98a5d98ec7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.132698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1116/2229] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:31 2022] PUT /identity/v3/projects/9f846197274944f7b2f7c46a9a78b8d7/users/1cdd046dfdb041158ed33e5902246921/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:31.145334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34d9ad-ccc7-4057-a00e-751fa0b94245 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.145541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34d9ad-ccc7-4057-a00e-751fa0b94245 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.145899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34d9ad-ccc7-4057-a00e-751fa0b94245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.186863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-8c34d9ad-ccc7-4057-a00e-751fa0b94245 None None] MFA Rules not processed for user `1cdd046dfdb041158ed33e5902246921`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:31.188118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8c34d9ad-ccc7-4057-a00e-751fa0b94245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:31.206012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963fde8-3425-47f7-b113-e9580ee1b51c admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.265770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:31.300053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b05f72-d235-4d72-9f02-f261f10d1069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.300792 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b05f72-d235-4d72-9f02-f261f10d1069 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.301284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b05f72-d235-4d72-9f02-f261f10d1069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.301711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b05f72-d235-4d72-9f02-f261f10d1069 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.302458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b05f72-d235-4d72-9f02-f261f10d1069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.304464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1117/2230] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:31 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:31.304604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b05f72-d235-4d72-9f02-f261f10d1069 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.305846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34d9ad-ccc7-4057-a00e-751fa0b94245 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.307728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8c34d9ad-ccc7-4057-a00e-751fa0b94245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:31.315087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:31.336890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1114/2231] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:31 2022] POST /identity/v3/users => generated 299 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:31.338222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b05f72-d235-4d72-9f02-f261f10d1069 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.342408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-75b05f72-d235-4d72-9f02-f261f10d1069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:31.397947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34d9ad-ccc7-4057-a00e-751fa0b94245 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.403251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54e71-e58c-48bf-9a32-7b4e20d0af51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.404479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54e71-e58c-48bf-9a32-7b4e20d0af51 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.404950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54e71-e58c-48bf-9a32-7b4e20d0af51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.405226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54e71-e58c-48bf-9a32-7b4e20d0af51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.407207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ffa54e71-e58c-48bf-9a32-7b4e20d0af51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:31.440407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b05f72-d235-4d72-9f02-f261f10d1069 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.444966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae5e39-d8b1-475b-9e67-1ada3ade0e79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.446182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae5e39-d8b1-475b-9e67-1ada3ade0e79 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.446539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae5e39-d8b1-475b-9e67-1ada3ade0e79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.446856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae5e39-d8b1-475b-9e67-1ada3ade0e79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.464331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae5e39-d8b1-475b-9e67-1ada3ade0e79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.464331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae5e39-d8b1-475b-9e67-1ada3ade0e79 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.480364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1115/2232] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:31 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:31.482009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae5e39-d8b1-475b-9e67-1ada3ade0e79 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.485168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa54e71-e58c-48bf-9a32-7b4e20d0af51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.488425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa54e71-e58c-48bf-9a32-7b4e20d0af51 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.498617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae5e39-d8b1-475b-9e67-1ada3ade0e79 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.501152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1118/2233] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:31.504909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bee4dc-9435-467b-9a78-d1a40d0745ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.505554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bee4dc-9435-467b-9a78-d1a40d0745ad admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.505832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bee4dc-9435-467b-9a78-d1a40d0745ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.506105 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bee4dc-9435-467b-9a78-d1a40d0745ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.506975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bee4dc-9435-467b-9a78-d1a40d0745ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.508734 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bee4dc-9435-467b-9a78-d1a40d0745ad admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.525454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1116/2234] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:31.530087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54e71-e58c-48bf-9a32-7b4e20d0af51 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.544161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54e71-e58c-48bf-9a32-7b4e20d0af51 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.550728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437149e8-d0c5-4e88-a5d4-a8958947b042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.552110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437149e8-d0c5-4e88-a5d4-a8958947b042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.552625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437149e8-d0c5-4e88-a5d4-a8958947b042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.553120 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437149e8-d0c5-4e88-a5d4-a8958947b042 admin admin] PATH_INFO: `/v3/projects/1b508ebc114c4e96baf25946bb8a5ac5/users/324e09697bdf4718862f3a3a05390f52/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.574900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437149e8-d0c5-4e88-a5d4-a8958947b042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b508ebc114c4e96baf25946bb8a5ac5, user_id=324e09697bdf4718862f3a3a05390f52, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.579710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437149e8-d0c5-4e88-a5d4-a8958947b042 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.596584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bee4dc-9435-467b-9a78-d1a40d0745ad admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.604968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1119/2235] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:31 2022] PUT /identity/v3/projects/1b508ebc114c4e96baf25946bb8a5ac5/users/324e09697bdf4718862f3a3a05390f52/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:31.608705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bee4dc-9435-467b-9a78-d1a40d0745ad admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.609476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437149e8-d0c5-4e88-a5d4-a8958947b042 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.667827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0546a19-2cb3-40c6-a7a7-7062a8bedd0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.668654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0546a19-2cb3-40c6-a7a7-7062a8bedd0f service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.668997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0546a19-2cb3-40c6-a7a7-7062a8bedd0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.669334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0546a19-2cb3-40c6-a7a7-7062a8bedd0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.688208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437149e8-d0c5-4e88-a5d4-a8958947b042 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.693153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8bddf-54e3-4aa4-96a7-26741689dfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.693783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8bddf-54e3-4aa4-96a7-26741689dfe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.694061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8bddf-54e3-4aa4-96a7-26741689dfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.694329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a8bddf-54e3-4aa4-96a7-26741689dfe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.695026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8bddf-54e3-4aa4-96a7-26741689dfe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.696679 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a8bddf-54e3-4aa4-96a7-26741689dfe8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.707311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1120/2236] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:31.711816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8bddf-54e3-4aa4-96a7-26741689dfe8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.722575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a8bddf-54e3-4aa4-96a7-26741689dfe8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.727300 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15910e7-d26b-4b2d-957c-67834ddd35fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.728060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15910e7-d26b-4b2d-957c-67834ddd35fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.728259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15910e7-d26b-4b2d-957c-67834ddd35fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.728565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15910e7-d26b-4b2d-957c-67834ddd35fe admin admin] PATH_INFO: `/v3/projects/1b508ebc114c4e96baf25946bb8a5ac5/users/324e09697bdf4718862f3a3a05390f52/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.729744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0546a19-2cb3-40c6-a7a7-7062a8bedd0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.730600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15910e7-d26b-4b2d-957c-67834ddd35fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b508ebc114c4e96baf25946bb8a5ac5, user_id=324e09697bdf4718862f3a3a05390f52, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:31.731273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0546a19-2cb3-40c6-a7a7-7062a8bedd0f service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.733321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15910e7-d26b-4b2d-957c-67834ddd35fe admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:31.735796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:31.742106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1117/2237] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:31.748477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1121/2238] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:31 2022] PUT /identity/v3/projects/1b508ebc114c4e96baf25946bb8a5ac5/users/324e09697bdf4718862f3a3a05390f52/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:31.759892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0ac36-f4b2-4847-aa9e-9f2dfcd90f32 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.760424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0ac36-f4b2-4847-aa9e-9f2dfcd90f32 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.760560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0ac36-f4b2-4847-aa9e-9f2dfcd90f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:31.791155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-03f0ac36-f4b2-4847-aa9e-9f2dfcd90f32 None None] MFA Rules not processed for user `324e09697bdf4718862f3a3a05390f52`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:31.792570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-03f0ac36-f4b2-4847-aa9e-9f2dfcd90f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:31.826979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:31.912878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1118/2239] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:31.926431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15910e7-d26b-4b2d-957c-67834ddd35fe admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:31.989509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15910e7-d26b-4b2d-957c-67834ddd35fe admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:31.994320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81942cd-99c5-4998-8aa3-26d6196f9486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:31.994988 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81942cd-99c5-4998-8aa3-26d6196f9486 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:31.995272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81942cd-99c5-4998-8aa3-26d6196f9486 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:31.995568 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81942cd-99c5-4998-8aa3-26d6196f9486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:32.066173 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81942cd-99c5-4998-8aa3-26d6196f9486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:32.068031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81942cd-99c5-4998-8aa3-26d6196f9486 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:32.078920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1122/2240] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:33.724083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0ac36-f4b2-4847-aa9e-9f2dfcd90f32 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:33.737680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0ac36-f4b2-4847-aa9e-9f2dfcd90f32 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:33.743936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aeaf7f-5be5-48cb-a2d1-0ce1f1e607ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:33.744636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeaf7f-5be5-48cb-a2d1-0ce1f1e607ef service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:33.744917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeaf7f-5be5-48cb-a2d1-0ce1f1e607ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:33.745204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aeaf7f-5be5-48cb-a2d1-0ce1f1e607ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:33.821870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aeaf7f-5be5-48cb-a2d1-0ce1f1e607ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:33.823388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aeaf7f-5be5-48cb-a2d1-0ce1f1e607ef service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:33.836355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1119/2241] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:34.597765 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81942cd-99c5-4998-8aa3-26d6196f9486 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:34.606912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81942cd-99c5-4998-8aa3-26d6196f9486 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:34.610853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751c549-8c42-4100-bf8b-4d43d6d56218 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:34.611460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751c549-8c42-4100-bf8b-4d43d6d56218 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:34.611798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751c549-8c42-4100-bf8b-4d43d6d56218 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:34.612132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751c549-8c42-4100-bf8b-4d43d6d56218 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:34.623385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751c549-8c42-4100-bf8b-4d43d6d56218 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:34.625169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751c549-8c42-4100-bf8b-4d43d6d56218 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:34.638922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1123/2242] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:36.119204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aeaf7f-5be5-48cb-a2d1-0ce1f1e607ef service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:36.121269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a9aeaf7f-5be5-48cb-a2d1-0ce1f1e607ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:36.235264 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aeaf7f-5be5-48cb-a2d1-0ce1f1e607ef service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:36.241499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:36.242356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:36.242685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:36.243423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] PATH_INFO: `/v3/users/ab1631136ea04f5381891accca29fc6c` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:36.245494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1631136ea04f5381891accca29fc6c)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:36.247147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:36.318599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1631136ea04f5381891accca29fc6c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:36.319606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1631136ea04f5381891accca29fc6c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:36.331165 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1631136ea04f5381891accca29fc6c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:36.339945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:36.354667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1631136ea04f5381891accca29fc6c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:36.364881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1631136ea04f5381891accca29fc6c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:36.372437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1120/2243] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:36 2022] DELETE /identity/v3/users/ab1631136ea04f5381891accca29fc6c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:36.377091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92590f92-bdf0-4c5b-b893-0080f65979af None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:36.377614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92590f92-bdf0-4c5b-b893-0080f65979af None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:36.377614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92590f92-bdf0-4c5b-b893-0080f65979af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:36.397509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-92590f92-bdf0-4c5b-b893-0080f65979af None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:36.398718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-92590f92-bdf0-4c5b-b893-0080f65979af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:36.461200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1124/2244] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:36.471055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:36.472619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:36.546270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cfa8c-1bb4-4e91-a6dd-ec89185e672a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:36.550262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f590da-bb3b-41cb-8c85-a3910062f722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:36.550893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f590da-bb3b-41cb-8c85-a3910062f722 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:36.551152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f590da-bb3b-41cb-8c85-a3910062f722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:36.551424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f590da-bb3b-41cb-8c85-a3910062f722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:36.561360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f590da-bb3b-41cb-8c85-a3910062f722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:36.562816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f590da-bb3b-41cb-8c85-a3910062f722 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:36.573012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1121/2245] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:36.856563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92590f92-bdf0-4c5b-b893-0080f65979af None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:36.858043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-92590f92-bdf0-4c5b-b893-0080f65979af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:36.872896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92590f92-bdf0-4c5b-b893-0080f65979af None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:36.878690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:36.879348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:36.879629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:36.879966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] PATH_INFO: `/v3/projects/83b59c6f908149568deed9de37a149ec` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:36.881352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83b59c6f908149568deed9de37a149ec)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:36.883717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:36.921444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:36.937439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83b59c6f908149568deed9de37a149ec', 'request_id': 'req-9e33aa51-4321-4756-9011-bfc608fa3175'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:36.943890 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83b59c6f908149568deed9de37a149ec', 'request_id': 'req-9e33aa51-4321-4756-9011-bfc608fa3175'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:36.951899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83b59c6f908149568deed9de37a149ec', 'request_id': 'req-9e33aa51-4321-4756-9011-bfc608fa3175'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:36.962914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] The token cache is being invalidate because project 83b59c6f908149568deed9de37a149ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:36.963205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:36.963847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:36.966085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1125/2246] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:36 2022] DELETE /identity/v3/projects/83b59c6f908149568deed9de37a149ec => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:36.970302 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f590da-bb3b-41cb-8c85-a3910062f722 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:36.980514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f590da-bb3b-41cb-8c85-a3910062f722 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:37.044142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:37.045093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:37.045380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:37.045676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] PATH_INFO: `/v3/users/3b918603f937441e82061b6cc4c58121` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:37.046883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b918603f937441e82061b6cc4c58121)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:37.048892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:37.101801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b918603f937441e82061b6cc4c58121'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.102421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b918603f937441e82061b6cc4c58121'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.111743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b918603f937441e82061b6cc4c58121'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.124599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b918603f937441e82061b6cc4c58121'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.131582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b918603f937441e82061b6cc4c58121'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.137567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1122/2247] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:36 2022] DELETE /identity/v3/users/3b918603f937441e82061b6cc4c58121 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:37.433537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:37.437128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:37.462344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33aa51-4321-4756-9011-bfc608fa3175 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:37.532074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:37.533065 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:37.533489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:37.533894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] PATH_INFO: `/v3/projects/86a46298236c4b60b83fb8d6b37127ee` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:37.535438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] RBAC: Authorizing `identity:delete_project(project_id=86a46298236c4b60b83fb8d6b37127ee)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:37.537462 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:37.586038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86a46298236c4b60b83fb8d6b37127ee', 'request_id': 'req-9057496b-b8e5-466e-b4c3-c6f849d1fbea'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.593763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86a46298236c4b60b83fb8d6b37127ee', 'request_id': 'req-9057496b-b8e5-466e-b4c3-c6f849d1fbea'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.603290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86a46298236c4b60b83fb8d6b37127ee', 'request_id': 'req-9057496b-b8e5-466e-b4c3-c6f849d1fbea'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.617170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] The token cache is being invalidate because project 86a46298236c4b60b83fb8d6b37127ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:37.617387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.618466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.620073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1126/2248] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:37 2022] DELETE /identity/v3/projects/86a46298236c4b60b83fb8d6b37127ee => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:37.624325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:37.625819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:37.634235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:37.640314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34080dd-e0bd-4142-ae1f-a1852fd2aaf1 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:37.702025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:37.702701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:37.702965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:37.703242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] PATH_INFO: `/v3/users/5f0e83344c4d4e0193bb32a86a5bf781` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:37.704478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0e83344c4d4e0193bb32a86a5bf781)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:37.706434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:37.759992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0e83344c4d4e0193bb32a86a5bf781'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.760656 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0e83344c4d4e0193bb32a86a5bf781'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.770004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0e83344c4d4e0193bb32a86a5bf781'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.782908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0e83344c4d4e0193bb32a86a5bf781'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.795653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8575bc88-0cc7-401c-bc46-e6005e11da4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0e83344c4d4e0193bb32a86a5bf781'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:37.802277 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1123/2249] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:37 2022] DELETE /identity/v3/users/5f0e83344c4d4e0193bb32a86a5bf781 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:38.093147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:38.094697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:38.117363 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057496b-b8e5-466e-b4c3-c6f849d1fbea admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:38.172643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:38.185923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:38.186572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.186846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.187117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] PATH_INFO: `/v3/projects/3ddcd13794b54a39ae3fe1cb6c471a60` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.188335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ddcd13794b54a39ae3fe1cb6c471a60)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:38.190245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:38.237212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ddcd13794b54a39ae3fe1cb6c471a60', 'request_id': 'req-6e9b42c0-c972-40cb-89eb-0373c7485c13'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:38.243664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ddcd13794b54a39ae3fe1cb6c471a60', 'request_id': 'req-6e9b42c0-c972-40cb-89eb-0373c7485c13'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:38.251099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ddcd13794b54a39ae3fe1cb6c471a60', 'request_id': 'req-6e9b42c0-c972-40cb-89eb-0373c7485c13'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:38.261874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] The token cache is being invalidate because project 3ddcd13794b54a39ae3fe1cb6c471a60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:38.262164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:38.262838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e9b42c0-c972-40cb-89eb-0373c7485c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:38.264409 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1127/2250] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:38 2022] DELETE /identity/v3/projects/3ddcd13794b54a39ae3fe1cb6c471a60 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:38.269497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf5bd5-092d-4992-b260-dab0fed5373a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.269963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf5bd5-092d-4992-b260-dab0fed5373a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.270050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf5bd5-092d-4992-b260-dab0fed5373a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.290547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2bbf5bd5-092d-4992-b260-dab0fed5373a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:38.291806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2bbf5bd5-092d-4992-b260-dab0fed5373a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:38.361344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1124/2251] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:38.369857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315b8b-bbf8-45f7-9f3c-3ad0109f00de None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.370149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315b8b-bbf8-45f7-9f3c-3ad0109f00de None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.370425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315b8b-bbf8-45f7-9f3c-3ad0109f00de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.396549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c0315b8b-bbf8-45f7-9f3c-3ad0109f00de None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:38.397792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c0315b8b-bbf8-45f7-9f3c-3ad0109f00de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:38.407789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1128/2252] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:38.412492 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf5bd5-092d-4992-b260-dab0fed5373a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:38.423617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf5bd5-092d-4992-b260-dab0fed5373a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:38.428711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6dd12-8d74-41c1-83e1-fb94635e7edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:38.429357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6dd12-8d74-41c1-83e1-fb94635e7edf admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.429636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6dd12-8d74-41c1-83e1-fb94635e7edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.429907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6dd12-8d74-41c1-83e1-fb94635e7edf admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.430613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6dd12-8d74-41c1-83e1-fb94635e7edf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:38.430921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6dd12-8d74-41c1-83e1-fb94635e7edf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:38.432562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf6dd12-8d74-41c1-83e1-fb94635e7edf admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:38.445875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1125/2253] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:38.450886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0315b8b-bbf8-45f7-9f3c-3ad0109f00de None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:38.461187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0315b8b-bbf8-45f7-9f3c-3ad0109f00de None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:38.466959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52be777-5e57-4bf2-b6ea-758d2deedb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:38.467587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52be777-5e57-4bf2-b6ea-758d2deedb62 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.467879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52be777-5e57-4bf2-b6ea-758d2deedb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.468220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52be777-5e57-4bf2-b6ea-758d2deedb62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.469090 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52be777-5e57-4bf2-b6ea-758d2deedb62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:38.471092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52be777-5e57-4bf2-b6ea-758d2deedb62 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:38.525789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1129/2254] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:38 2022] POST /identity/v3/projects => generated 363 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:38.530749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6dd12-8d74-41c1-83e1-fb94635e7edf admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:38.541169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6dd12-8d74-41c1-83e1-fb94635e7edf admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:38.546561 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:38.602676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356eddb-f923-455b-be61-c60e5c522749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:38.603388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356eddb-f923-455b-be61-c60e5c522749 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.603733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356eddb-f923-455b-be61-c60e5c522749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.604076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356eddb-f923-455b-be61-c60e5c522749 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.604846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356eddb-f923-455b-be61-c60e5c522749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:38.606843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356eddb-f923-455b-be61-c60e5c522749 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:38.633001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1126/2255] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:38 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:38.634878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52be777-5e57-4bf2-b6ea-758d2deedb62 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:38.645539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52be777-5e57-4bf2-b6ea-758d2deedb62 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:38.650937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc321ff-f91e-4e10-bee3-cbaf96575d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:38.651566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc321ff-f91e-4e10-bee3-cbaf96575d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.651918 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc321ff-f91e-4e10-bee3-cbaf96575d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.652225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc321ff-f91e-4e10-bee3-cbaf96575d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.652974 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc321ff-f91e-4e10-bee3-cbaf96575d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:38.654768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc321ff-f91e-4e10-bee3-cbaf96575d01 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:38.666094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1130/2256] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:38.671586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356eddb-f923-455b-be61-c60e5c522749 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:38.683119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356eddb-f923-455b-be61-c60e5c522749 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:38.688238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79e393-9a0d-471c-8737-72c3cbebd2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:38.688882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e393-9a0d-471c-8737-72c3cbebd2bf admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.689270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e393-9a0d-471c-8737-72c3cbebd2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.689544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e393-9a0d-471c-8737-72c3cbebd2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.690249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79e393-9a0d-471c-8737-72c3cbebd2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:38.691984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79e393-9a0d-471c-8737-72c3cbebd2bf admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:38.702912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1127/2257] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:38.708759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc321ff-f91e-4e10-bee3-cbaf96575d01 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:38.719071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc321ff-f91e-4e10-bee3-cbaf96575d01 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:38.724962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690e06-688b-4ca9-9757-d4c63650e940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:38.725624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690e06-688b-4ca9-9757-d4c63650e940 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.725900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690e06-688b-4ca9-9757-d4c63650e940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.726175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9690e06-688b-4ca9-9757-d4c63650e940 admin admin] PATH_INFO: `/v3/projects/e263806394b64815ac8677c51a8b893a/users/709c635becce4f9aae0eeb81ccace346/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.742578 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9690e06-688b-4ca9-9757-d4c63650e940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e263806394b64815ac8677c51a8b893a, user_id=709c635becce4f9aae0eeb81ccace346, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:38.745692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9690e06-688b-4ca9-9757-d4c63650e940 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:38.762449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1131/2258] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:38 2022] PUT /identity/v3/projects/e263806394b64815ac8677c51a8b893a/users/709c635becce4f9aae0eeb81ccace346/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:38.773345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d23a4b-e9e4-4489-9994-07201cd890c9 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.773618 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d23a4b-e9e4-4489-9994-07201cd890c9 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.773885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d23a4b-e9e4-4489-9994-07201cd890c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.810391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-62d23a4b-e9e4-4489-9994-07201cd890c9 None None] MFA Rules not processed for user `709c635becce4f9aae0eeb81ccace346`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:38.811598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-62d23a4b-e9e4-4489-9994-07201cd890c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:38.878437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1128/2259] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:38 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:38.890425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690e06-688b-4ca9-9757-d4c63650e940 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:38.892281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c9690e06-688b-4ca9-9757-d4c63650e940 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:38.951338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9690e06-688b-4ca9-9757-d4c63650e940 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:38.956550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205e2d1-4421-47b5-ad80-caab47a1c9bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:38.957689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205e2d1-4421-47b5-ad80-caab47a1c9bb service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:38.958202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205e2d1-4421-47b5-ad80-caab47a1c9bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:38.958705 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205e2d1-4421-47b5-ad80-caab47a1c9bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:38.973641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205e2d1-4421-47b5-ad80-caab47a1c9bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:38.975082 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205e2d1-4421-47b5-ad80-caab47a1c9bb service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:38.985395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1132/2260] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:38 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:44.243859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d23a4b-e9e4-4489-9994-07201cd890c9 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:44.245389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-62d23a4b-e9e4-4489-9994-07201cd890c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:44.334463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d23a4b-e9e4-4489-9994-07201cd890c9 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:44.351577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:44.351577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:44.351577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:44.351577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] PATH_INFO: `/v3/users/2e8e0e68c64b488b8c08cc743385e873` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:44.353520 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e8e0e68c64b488b8c08cc743385e873)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:44.355791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:44.378114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:44.405152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e8e0e68c64b488b8c08cc743385e873'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:44.405817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e8e0e68c64b488b8c08cc743385e873'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:44.415173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e8e0e68c64b488b8c08cc743385e873'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:44.428858 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e8e0e68c64b488b8c08cc743385e873'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:44.445378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e8e0e68c64b488b8c08cc743385e873'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:44.453201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1129/2261] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:44 2022] DELETE /identity/v3/users/2e8e0e68c64b488b8c08cc743385e873 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:44.457634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad410d-f148-4bba-aa40-aa46fb5e01db None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:44.458219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad410d-f148-4bba-aa40-aa46fb5e01db None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:44.458219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad410d-f148-4bba-aa40-aa46fb5e01db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:44.484008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-dfad410d-f148-4bba-aa40-aa46fb5e01db None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:44.485687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-dfad410d-f148-4bba-aa40-aa46fb5e01db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:44.499545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:44.573081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1133/2262] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:44.584347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:44.586357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:44.649328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f1dcf-3bda-4df8-a236-7d60e76eb08c admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:44.653082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d840a-c47f-4a27-9964-7bd516adf452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:44.653784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d840a-c47f-4a27-9964-7bd516adf452 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:44.654147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d840a-c47f-4a27-9964-7bd516adf452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:44.654480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d840a-c47f-4a27-9964-7bd516adf452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:44.664602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d840a-c47f-4a27-9964-7bd516adf452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:44.666087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d840a-c47f-4a27-9964-7bd516adf452 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:44.679606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1130/2263] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:44.971813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad410d-f148-4bba-aa40-aa46fb5e01db None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:44.973687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-dfad410d-f148-4bba-aa40-aa46fb5e01db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:44.992786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad410d-f148-4bba-aa40-aa46fb5e01db None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:45.001114 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:45.002084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:45.002457 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:45.002798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] PATH_INFO: `/v3/projects/37eab4b79bda465186e12a8d8ddf58e2` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:45.004709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37eab4b79bda465186e12a8d8ddf58e2)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:45.007444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:45.087305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37eab4b79bda465186e12a8d8ddf58e2', 'request_id': 'req-c0078e20-1f34-4185-9bfd-011e01baecc9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.098576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37eab4b79bda465186e12a8d8ddf58e2', 'request_id': 'req-c0078e20-1f34-4185-9bfd-011e01baecc9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.110083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37eab4b79bda465186e12a8d8ddf58e2', 'request_id': 'req-c0078e20-1f34-4185-9bfd-011e01baecc9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.127110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] The token cache is being invalidate because project 37eab4b79bda465186e12a8d8ddf58e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:45.127649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.128069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.130044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1134/2264] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:44 2022] DELETE /identity/v3/projects/37eab4b79bda465186e12a8d8ddf58e2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:45.134006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d840a-c47f-4a27-9964-7bd516adf452 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:45.152394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d840a-c47f-4a27-9964-7bd516adf452 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:45.215896 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:45.216782 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:45.216990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:45.217157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] PATH_INFO: `/v3/users/cceeda2b1d5d45339ccb2fb0f13cbbf7` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:45.218638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] RBAC: Authorizing `identity:delete_user(user_id=cceeda2b1d5d45339ccb2fb0f13cbbf7)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:45.220758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:45.275543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:45.282315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cceeda2b1d5d45339ccb2fb0f13cbbf7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.283038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cceeda2b1d5d45339ccb2fb0f13cbbf7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.291949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cceeda2b1d5d45339ccb2fb0f13cbbf7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.305125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cceeda2b1d5d45339ccb2fb0f13cbbf7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.318367 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cceeda2b1d5d45339ccb2fb0f13cbbf7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.325270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1131/2265] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:45 2022] DELETE /identity/v3/users/cceeda2b1d5d45339ccb2fb0f13cbbf7 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:45.685634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:45.689449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:45.727609 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0078e20-1f34-4185-9bfd-011e01baecc9 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:45.819470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:45.820184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:45.820499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:45.820973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] PATH_INFO: `/v3/projects/d8880914a74748e79395722a9be4f5b4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:45.822160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8880914a74748e79395722a9be4f5b4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:45.824112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:45.859910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:45.862875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:45.876864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6797e8de-625c-4fcf-856a-6bb8abc0ffde admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:45.886665 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:45.893747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8880914a74748e79395722a9be4f5b4', 'request_id': 'req-e75f51b8-a4c8-487d-b5f9-8fb3db198882'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.908637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8880914a74748e79395722a9be4f5b4', 'request_id': 'req-e75f51b8-a4c8-487d-b5f9-8fb3db198882'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.924626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8880914a74748e79395722a9be4f5b4', 'request_id': 'req-e75f51b8-a4c8-487d-b5f9-8fb3db198882'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.940983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] The token cache is being invalidate because project d8880914a74748e79395722a9be4f5b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:45.941389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.942411 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:45.944814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1135/2266] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:45 2022] DELETE /identity/v3/projects/d8880914a74748e79395722a9be4f5b4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:45.949409 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:45.951251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:45.957900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:45.958568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:45.958853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:45.959230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] PATH_INFO: `/v3/users/f7dff0370f794a93bfc61948e1845ec1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:45.960662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7dff0370f794a93bfc61948e1845ec1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:45.962702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:45.969303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f51b8-a4c8-487d-b5f9-8fb3db198882 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:45.977634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:45.978627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:45.979590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:45.980052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] PATH_INFO: `/v3/users/324e09697bdf4718862f3a3a05390f52` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:45.982014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324e09697bdf4718862f3a3a05390f52)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:45.986223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:46.033623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7dff0370f794a93bfc61948e1845ec1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.034595 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7dff0370f794a93bfc61948e1845ec1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.043584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7dff0370f794a93bfc61948e1845ec1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.045472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324e09697bdf4718862f3a3a05390f52'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.057018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7dff0370f794a93bfc61948e1845ec1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.059538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324e09697bdf4718862f3a3a05390f52'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.065772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-42d19543-892c-4327-a3d3-065e18d06320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7dff0370f794a93bfc61948e1845ec1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.068637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324e09697bdf4718862f3a3a05390f52'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.069372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324e09697bdf4718862f3a3a05390f52'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.070271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324e09697bdf4718862f3a3a05390f52'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.075195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1132/2267] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:45 2022] DELETE /identity/v3/users/f7dff0370f794a93bfc61948e1845ec1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:46.076755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7fc69-9c7e-4b73-9c49-82b700fe763e None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:46.077042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7fc69-9c7e-4b73-9c49-82b700fe763e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:46.077325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7fc69-9c7e-4b73-9c49-82b700fe763e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:46.089024 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1136/2268] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:45 2022] DELETE /identity/v3/users/324e09697bdf4718862f3a3a05390f52 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:46.098702 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-17f7fc69-9c7e-4b73-9c49-82b700fe763e None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:46.100028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-17f7fc69-9c7e-4b73-9c49-82b700fe763e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:46.145620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:46.188343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1133/2269] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:46.196682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:46.197819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:46.311382 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555edf8-bf8b-464b-b3da-f876b62ac978 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:46.316171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8dac5-c5b5-4392-af9e-f21cf93bd7c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:46.316865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8dac5-c5b5-4392-af9e-f21cf93bd7c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:46.317148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8dac5-c5b5-4392-af9e-f21cf93bd7c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:46.317424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8dac5-c5b5-4392-af9e-f21cf93bd7c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:46.322744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7fc69-9c7e-4b73-9c49-82b700fe763e None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:46.324879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-17f7fc69-9c7e-4b73-9c49-82b700fe763e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:46.339716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8dac5-c5b5-4392-af9e-f21cf93bd7c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:46.341249 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7fc69-9c7e-4b73-9c49-82b700fe763e None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:46.342209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8dac5-c5b5-4392-af9e-f21cf93bd7c2 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:46.346298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:46.346980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:46.347257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:46.347531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] PATH_INFO: `/v3/users/26eff167003f47188d6c922735e1653f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:46.348786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26eff167003f47188d6c922735e1653f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:46.350740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:46.359043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1137/2270] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:46.425579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26eff167003f47188d6c922735e1653f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.426404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26eff167003f47188d6c922735e1653f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.436552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26eff167003f47188d6c922735e1653f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.460038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26eff167003f47188d6c922735e1653f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.478606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-737ce0d1-87ef-4687-bf1e-1ba041e26e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26eff167003f47188d6c922735e1653f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.496384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1134/2271] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:46 2022] DELETE /identity/v3/users/26eff167003f47188d6c922735e1653f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:46.497037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8dac5-c5b5-4392-af9e-f21cf93bd7c2 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:46.498033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c510e2-08cb-4369-bb9f-271ea7bbedbd None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:46.498360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c510e2-08cb-4369-bb9f-271ea7bbedbd None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:46.498690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b3c8dac5-c5b5-4392-af9e-f21cf93bd7c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:46.498842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c510e2-08cb-4369-bb9f-271ea7bbedbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:46.513835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8dac5-c5b5-4392-af9e-f21cf93bd7c2 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:46.521627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c2c510e2-08cb-4369-bb9f-271ea7bbedbd None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:46.524160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c2c510e2-08cb-4369-bb9f-271ea7bbedbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:46.579186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:46.601553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:46.602354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:46.602634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:46.602909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] PATH_INFO: `/v3/projects/1b508ebc114c4e96baf25946bb8a5ac5` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:46.604667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b508ebc114c4e96baf25946bb8a5ac5)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:46.606197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1135/2272] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:46.609147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:46.615993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c510e2-08cb-4369-bb9f-271ea7bbedbd None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:46.618609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c2c510e2-08cb-4369-bb9f-271ea7bbedbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:46.691049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b508ebc114c4e96baf25946bb8a5ac5', 'request_id': 'req-887eb76f-3092-494c-9579-0ebfe585a705'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.701135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b508ebc114c4e96baf25946bb8a5ac5', 'request_id': 'req-887eb76f-3092-494c-9579-0ebfe585a705'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.701924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c510e2-08cb-4369-bb9f-271ea7bbedbd None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:46.705805 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:46.723713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b508ebc114c4e96baf25946bb8a5ac5', 'request_id': 'req-887eb76f-3092-494c-9579-0ebfe585a705'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.748200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] The token cache is being invalidate because project 1b508ebc114c4e96baf25946bb8a5ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:46.749484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.751971 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-887eb76f-3092-494c-9579-0ebfe585a705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:46.755437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1138/2273] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:46 2022] DELETE /identity/v3/projects/1b508ebc114c4e96baf25946bb8a5ac5 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:46.760665 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3eaa4-fbf7-489a-aa5d-6caf300cfc3c None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:46.761217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3eaa4-fbf7-489a-aa5d-6caf300cfc3c None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:46.761569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3eaa4-fbf7-489a-aa5d-6caf300cfc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:46.774825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cdf20-5d32-42e5-a1cb-290d445dd789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:46.775516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cdf20-5d32-42e5-a1cb-290d445dd789 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:46.775796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cdf20-5d32-42e5-a1cb-290d445dd789 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:46.776119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cdf20-5d32-42e5-a1cb-290d445dd789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:46.778431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d11cdf20-5d32-42e5-a1cb-290d445dd789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:46.865929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e1f3eaa4-fbf7-489a-aa5d-6caf300cfc3c None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:46.873361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e1f3eaa4-fbf7-489a-aa5d-6caf300cfc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:46.908182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1139/2274] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:46.914664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cdf20-5d32-42e5-a1cb-290d445dd789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:46.918177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22049f-c3de-41e8-a95e-603134a64173 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:46.918647 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22049f-c3de-41e8-a95e-603134a64173 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:46.919302 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22049f-c3de-41e8-a95e-603134a64173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:46.927298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11cdf20-5d32-42e5-a1cb-290d445dd789 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:46.966511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1136/2275] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:46.977677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-7c22049f-c3de-41e8-a95e-603134a64173 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:46.979019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7c22049f-c3de-41e8-a95e-603134a64173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:46.987789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1140/2276] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:46.994325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cdf20-5d32-42e5-a1cb-290d445dd789 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.009186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cdf20-5d32-42e5-a1cb-290d445dd789 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.017171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d24eb2-cd9c-4f16-a081-7de85a95fe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.017510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d24eb2-cd9c-4f16-a081-7de85a95fe37 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.017836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d24eb2-cd9c-4f16-a081-7de85a95fe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.018682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d24eb2-cd9c-4f16-a081-7de85a95fe37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.019153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d24eb2-cd9c-4f16-a081-7de85a95fe37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:47.019612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d24eb2-cd9c-4f16-a081-7de85a95fe37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.021696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d24eb2-cd9c-4f16-a081-7de85a95fe37 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.042016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22049f-c3de-41e8-a95e-603134a64173 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.046540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1137/2277] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:47.047594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7c22049f-c3de-41e8-a95e-603134a64173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:47.057616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d24eb2-cd9c-4f16-a081-7de85a95fe37 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.072686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22049f-c3de-41e8-a95e-603134a64173 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.082361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.083180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.083298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.083540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] PATH_INFO: `/v3/projects/bbfb9b6e55a149cab7cddf262353e0cc` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.085512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbfb9b6e55a149cab7cddf262353e0cc)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.087555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.101757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d24eb2-cd9c-4f16-a081-7de85a95fe37 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.113708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b5b8d-0a7b-4442-8871-d67430fa39fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.117481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b5b8d-0a7b-4442-8871-d67430fa39fa admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.120195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b5b8d-0a7b-4442-8871-d67430fa39fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.120718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b5b8d-0a7b-4442-8871-d67430fa39fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.121516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b5b8d-0a7b-4442-8871-d67430fa39fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.127353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b5b8d-0a7b-4442-8871-d67430fa39fa admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.167407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1138/2278] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:47 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:47.171955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b5b8d-0a7b-4442-8871-d67430fa39fa admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.173887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbfb9b6e55a149cab7cddf262353e0cc', 'request_id': 'req-89c59738-530c-45cb-9d50-465922bf2ac7'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.184282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbfb9b6e55a149cab7cddf262353e0cc', 'request_id': 'req-89c59738-530c-45cb-9d50-465922bf2ac7'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.197810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbfb9b6e55a149cab7cddf262353e0cc', 'request_id': 'req-89c59738-530c-45cb-9d50-465922bf2ac7'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.214260 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] The token cache is being invalidate because project bbfb9b6e55a149cab7cddf262353e0cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:47.214538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.215404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.217273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1141/2279] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:47 2022] DELETE /identity/v3/projects/bbfb9b6e55a149cab7cddf262353e0cc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:47.221087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.224743 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:47.268192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b5b8d-0a7b-4442-8871-d67430fa39fa admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.278857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8b65d-c695-47f6-bca7-088d9a3e4f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.280631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8b65d-c695-47f6-bca7-088d9a3e4f64 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.281232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8b65d-c695-47f6-bca7-088d9a3e4f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.281597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e8b65d-c695-47f6-bca7-088d9a3e4f64 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.283035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8b65d-c695-47f6-bca7-088d9a3e4f64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.286409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e8b65d-c695-47f6-bca7-088d9a3e4f64 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.291415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:47.312369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c59738-530c-45cb-9d50-465922bf2ac7 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.313852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1139/2280] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:47 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:47.317661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8b65d-c695-47f6-bca7-088d9a3e4f64 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.319698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e5e8b65d-c695-47f6-bca7-088d9a3e4f64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:47.322183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.322620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.322904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.323368 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] PATH_INFO: `/v3/users/1cdd046dfdb041158ed33e5902246921` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.325506 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cdd046dfdb041158ed33e5902246921)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.327659 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.332016 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e8b65d-c695-47f6-bca7-088d9a3e4f64 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.337820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97495cdd-4dd6-4eee-93f3-482f0a7b1428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.338784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97495cdd-4dd6-4eee-93f3-482f0a7b1428 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.339063 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97495cdd-4dd6-4eee-93f3-482f0a7b1428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.339332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97495cdd-4dd6-4eee-93f3-482f0a7b1428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.340226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97495cdd-4dd6-4eee-93f3-482f0a7b1428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.342348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97495cdd-4dd6-4eee-93f3-482f0a7b1428 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.349475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:47.359226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1140/2281] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:47.360264 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97495cdd-4dd6-4eee-93f3-482f0a7b1428 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.372112 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97495cdd-4dd6-4eee-93f3-482f0a7b1428 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.379381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affe65c-28d7-465c-8f91-32962cbe3d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.380070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affe65c-28d7-465c-8f91-32962cbe3d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.380390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affe65c-28d7-465c-8f91-32962cbe3d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.380712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affe65c-28d7-465c-8f91-32962cbe3d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.381424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2affe65c-28d7-465c-8f91-32962cbe3d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.383180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2affe65c-28d7-465c-8f91-32962cbe3d86 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.390879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cdd046dfdb041158ed33e5902246921'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.395657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1141/2282] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:47.400089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affe65c-28d7-465c-8f91-32962cbe3d86 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.411738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cdd046dfdb041158ed33e5902246921'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.416092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affe65c-28d7-465c-8f91-32962cbe3d86 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.424213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c32ce-c851-4346-991a-ee2d66261599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.424213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c32ce-c851-4346-991a-ee2d66261599 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.424213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c32ce-c851-4346-991a-ee2d66261599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.424213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c32ce-c851-4346-991a-ee2d66261599 admin admin] PATH_INFO: `/v3/projects/b0422eb0a2a74ae099b22570c84afbbb/users/f82ce1e8ee7d441e9365de3caf8f6c6f/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.427063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cdd046dfdb041158ed33e5902246921'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.428150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cdd046dfdb041158ed33e5902246921'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.429122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cdd046dfdb041158ed33e5902246921'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.442622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c32ce-c851-4346-991a-ee2d66261599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0422eb0a2a74ae099b22570c84afbbb, user_id=f82ce1e8ee7d441e9365de3caf8f6c6f, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.447336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1142/2283] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:47 2022] DELETE /identity/v3/users/1cdd046dfdb041158ed33e5902246921 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:47.450036 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c32ce-c851-4346-991a-ee2d66261599 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.470433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.472621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:47.474069 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1142/2284] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:47 2022] PUT /identity/v3/projects/b0422eb0a2a74ae099b22570c84afbbb/users/f82ce1e8ee7d441e9365de3caf8f6c6f/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:47.479885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2a9ec-d93b-40a0-92a4-a4ccf37071c1 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.481028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2a9ec-d93b-40a0-92a4-a4ccf37071c1 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.481828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2a9ec-d93b-40a0-92a4-a4ccf37071c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.529276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-42c2a9ec-d93b-40a0-92a4-a4ccf37071c1 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:47.532530 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-42c2a9ec-d93b-40a0-92a4-a4ccf37071c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:47.587089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6056709-a9b9-4aa8-898d-8016ddf58501 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.611777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1143/2285] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:47.639864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2a9ec-d93b-40a0-92a4-a4ccf37071c1 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.645518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-42c2a9ec-d93b-40a0-92a4-a4ccf37071c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:47.669784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.670856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.671466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.671946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] PATH_INFO: `/v3/projects/a0bf742a930e4c0bbc736b33cd354e2c` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.674468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0bf742a930e4c0bbc736b33cd354e2c)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.677803 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.701685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:47.737070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c2a9ec-d93b-40a0-92a4-a4ccf37071c1 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.747712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0bf742a930e4c0bbc736b33cd354e2c', 'request_id': 'req-6e5678be-8ca6-4490-b839-ed6856e78dcc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.762427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0bf742a930e4c0bbc736b33cd354e2c', 'request_id': 'req-6e5678be-8ca6-4490-b839-ed6856e78dcc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.772677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0bf742a930e4c0bbc736b33cd354e2c', 'request_id': 'req-6e5678be-8ca6-4490-b839-ed6856e78dcc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.785528 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] The token cache is being invalidate because project a0bf742a930e4c0bbc736b33cd354e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:47.785819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.786499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.788401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1143/2286] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:47 2022] DELETE /identity/v3/projects/a0bf742a930e4c0bbc736b33cd354e2c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:47.789421 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:47.790765 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:47.838381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7fd61-68d5-4ce3-8dbc-14e78883410c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.839080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7fd61-68d5-4ce3-8dbc-14e78883410c service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.839364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7fd61-68d5-4ce3-8dbc-14e78883410c service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.839666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7fd61-68d5-4ce3-8dbc-14e78883410c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.841972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-fee7fd61-68d5-4ce3-8dbc-14e78883410c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:47.874186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5678be-8ca6-4490-b839-ed6856e78dcc admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:47.879304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:47.880107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.880427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.880703 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] PATH_INFO: `/v3/projects/9f846197274944f7b2f7c46a9a78b8d7` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.881951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f846197274944f7b2f7c46a9a78b8d7)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.882109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:47.884126 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.917790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee7fd61-68d5-4ce3-8dbc-14e78883410c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:47.919258 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee7fd61-68d5-4ce3-8dbc-14e78883410c service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:47.932312 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1144/2287] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:47.938125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf073a-7573-4b07-8744-67066130bb08 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.938564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf073a-7573-4b07-8744-67066130bb08 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.939035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf073a-7573-4b07-8744-67066130bb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:47.945885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f846197274944f7b2f7c46a9a78b8d7', 'request_id': 'req-22c4246c-7d10-44e8-b239-7683742935a9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.952402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f846197274944f7b2f7c46a9a78b8d7', 'request_id': 'req-22c4246c-7d10-44e8-b239-7683742935a9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.962647 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f846197274944f7b2f7c46a9a78b8d7', 'request_id': 'req-22c4246c-7d10-44e8-b239-7683742935a9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.962970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-73cf073a-7573-4b07-8744-67066130bb08 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:47.964216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-73cf073a-7573-4b07-8744-67066130bb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:47.976220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] The token cache is being invalidate because project 9f846197274944f7b2f7c46a9a78b8d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:47.976660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.977587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-22c4246c-7d10-44e8-b239-7683742935a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:47.979211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1144/2288] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:47 2022] DELETE /identity/v3/projects/9f846197274944f7b2f7c46a9a78b8d7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:47.984453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea008aec-c16e-4925-b6f9-3c2c12150a84 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:47.984748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea008aec-c16e-4925-b6f9-3c2c12150a84 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:47.985023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea008aec-c16e-4925-b6f9-3c2c12150a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.010775 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ea008aec-c16e-4925-b6f9-3c2c12150a84 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:48.012615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ea008aec-c16e-4925-b6f9-3c2c12150a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:48.043184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:48.044916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1145/2289] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:48.053540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099643-7a31-4161-aabc-687b37ae1047 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.053872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099643-7a31-4161-aabc-687b37ae1047 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.054279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099643-7a31-4161-aabc-687b37ae1047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.075832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-e2099643-7a31-4161-aabc-687b37ae1047 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:48.077111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e2099643-7a31-4161-aabc-687b37ae1047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:48.094092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1146/2290] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:48.099447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2099643-7a31-4161-aabc-687b37ae1047 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.107691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1145/2291] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:48.113209 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2099643-7a31-4161-aabc-687b37ae1047 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.117115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d3ce-ee3e-4024-8cfe-5ae001a865cf None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.117464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d3ce-ee3e-4024-8cfe-5ae001a865cf None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.117852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2d3ce-ee3e-4024-8cfe-5ae001a865cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.121498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ab955-5415-49b2-b954-49b96912a391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.122961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ab955-5415-49b2-b954-49b96912a391 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.123384 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ab955-5415-49b2-b954-49b96912a391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.123667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ab955-5415-49b2-b954-49b96912a391 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.124597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ab955-5415-49b2-b954-49b96912a391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:48.124977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ab955-5415-49b2-b954-49b96912a391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.126959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ab955-5415-49b2-b954-49b96912a391 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.142964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-67e2d3ce-ee3e-4024-8cfe-5ae001a865cf None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:48.143563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1147/2292] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:48.144692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-67e2d3ce-ee3e-4024-8cfe-5ae001a865cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:48.148561 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ab955-5415-49b2-b954-49b96912a391 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.164029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1146/2293] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:48.164827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2d3ce-ee3e-4024-8cfe-5ae001a865cf None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.176023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2d3ce-ee3e-4024-8cfe-5ae001a865cf None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.181185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ab955-5415-49b2-b954-49b96912a391 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.184081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2380dd-e8b9-4ef0-a74e-f85d60ecc478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.184874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2380dd-e8b9-4ef0-a74e-f85d60ecc478 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.185307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2380dd-e8b9-4ef0-a74e-f85d60ecc478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.185604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2380dd-e8b9-4ef0-a74e-f85d60ecc478 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.187337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2380dd-e8b9-4ef0-a74e-f85d60ecc478 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:48.187607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2380dd-e8b9-4ef0-a74e-f85d60ecc478 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.189600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974acf97-a192-4b89-ae2f-2b7cfaf598cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.189736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2380dd-e8b9-4ef0-a74e-f85d60ecc478 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.190733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974acf97-a192-4b89-ae2f-2b7cfaf598cb admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.191142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974acf97-a192-4b89-ae2f-2b7cfaf598cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.191422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974acf97-a192-4b89-ae2f-2b7cfaf598cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.192260 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974acf97-a192-4b89-ae2f-2b7cfaf598cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.195594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974acf97-a192-4b89-ae2f-2b7cfaf598cb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.209647 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1147/2294] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:06:48.210773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2380dd-e8b9-4ef0-a74e-f85d60ecc478 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.224045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2380dd-e8b9-4ef0-a74e-f85d60ecc478 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.230084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44fc6b-6177-48c0-b0b8-a2b2bfba6115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.231030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44fc6b-6177-48c0-b0b8-a2b2bfba6115 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.231383 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44fc6b-6177-48c0-b0b8-a2b2bfba6115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.231838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44fc6b-6177-48c0-b0b8-a2b2bfba6115 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.232856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44fc6b-6177-48c0-b0b8-a2b2bfba6115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.235503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44fc6b-6177-48c0-b0b8-a2b2bfba6115 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.267775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1148/2295] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:48.269089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974acf97-a192-4b89-ae2f-2b7cfaf598cb admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.280805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1148/2296] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:48.285435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44fc6b-6177-48c0-b0b8-a2b2bfba6115 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.349856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974acf97-a192-4b89-ae2f-2b7cfaf598cb admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.394799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:48.423310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44fc6b-6177-48c0-b0b8-a2b2bfba6115 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.429794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f3ef2-0e0e-417a-8c5e-ad5344ae64c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.430437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f3ef2-0e0e-417a-8c5e-ad5344ae64c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.430807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f3ef2-0e0e-417a-8c5e-ad5344ae64c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.431444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f3ef2-0e0e-417a-8c5e-ad5344ae64c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.432198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f3ef2-0e0e-417a-8c5e-ad5344ae64c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.435000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f3ef2-0e0e-417a-8c5e-ad5344ae64c8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.444786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3de9e1-92cb-4cd3-ad66-2a40c7305a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.446354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3de9e1-92cb-4cd3-ad66-2a40c7305a15 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.447726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3de9e1-92cb-4cd3-ad66-2a40c7305a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.448579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3de9e1-92cb-4cd3-ad66-2a40c7305a15 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.449296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3de9e1-92cb-4cd3-ad66-2a40c7305a15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.453990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3de9e1-92cb-4cd3-ad66-2a40c7305a15 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.467054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1149/2297] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:48.471939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f3ef2-0e0e-417a-8c5e-ad5344ae64c8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.498195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f3ef2-0e0e-417a-8c5e-ad5344ae64c8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.502832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1149/2298] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/users => generated 286 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:48.508527 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3de9e1-92cb-4cd3-ad66-2a40c7305a15 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.520426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3de9e1-92cb-4cd3-ad66-2a40c7305a15 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.521218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7cad4-a60d-49ed-97be-d9a98bef5323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.521990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7cad4-a60d-49ed-97be-d9a98bef5323 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.522341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7cad4-a60d-49ed-97be-d9a98bef5323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.522708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f7cad4-a60d-49ed-97be-d9a98bef5323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.523659 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7cad4-a60d-49ed-97be-d9a98bef5323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.525448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f7cad4-a60d-49ed-97be-d9a98bef5323 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.527184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bd395-2af6-41d9-810f-895f86ae0239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.528042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bd395-2af6-41d9-810f-895f86ae0239 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.528543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bd395-2af6-41d9-810f-895f86ae0239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.528991 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bd395-2af6-41d9-810f-895f86ae0239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.530071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035bd395-2af6-41d9-810f-895f86ae0239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.532442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035bd395-2af6-41d9-810f-895f86ae0239 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.544691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1150/2299] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:48.546568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1150/2300] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:48.551122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7cad4-a60d-49ed-97be-d9a98bef5323 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.551548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bd395-2af6-41d9-810f-895f86ae0239 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.565312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f7cad4-a60d-49ed-97be-d9a98bef5323 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.565559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035bd395-2af6-41d9-810f-895f86ae0239 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.573475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f601b99-d70f-4b07-b49c-ef6d659f4159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.574043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e8f3f-e001-4071-86e8-d44b6500de96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.574228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f601b99-d70f-4b07-b49c-ef6d659f4159 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.574575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f601b99-d70f-4b07-b49c-ef6d659f4159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.575073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f601b99-d70f-4b07-b49c-ef6d659f4159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.575860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e8f3f-e001-4071-86e8-d44b6500de96 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.576359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e8f3f-e001-4071-86e8-d44b6500de96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.576872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e8f3f-e001-4071-86e8-d44b6500de96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.577187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f601b99-d70f-4b07-b49c-ef6d659f4159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.578540 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e8f3f-e001-4071-86e8-d44b6500de96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.579317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f601b99-d70f-4b07-b49c-ef6d659f4159 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.581144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e8f3f-e001-4071-86e8-d44b6500de96 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.599786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1151/2301] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:48.600908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e8f3f-e001-4071-86e8-d44b6500de96 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.602831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1151/2302] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:48.608140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f601b99-d70f-4b07-b49c-ef6d659f4159 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.613478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e8f3f-e001-4071-86e8-d44b6500de96 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.619596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c531d-2f5e-4608-8bd3-32b6b6b031e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.620488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c531d-2f5e-4608-8bd3-32b6b6b031e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.620768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c531d-2f5e-4608-8bd3-32b6b6b031e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.621040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c531d-2f5e-4608-8bd3-32b6b6b031e1 admin admin] PATH_INFO: `/v3/projects/2a5cc1d2dee4412eb37b794156fdcb83/users/24f34c5beaad416188a32cf0860dd93e/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.628543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f601b99-d70f-4b07-b49c-ef6d659f4159 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.636739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eabf677-a632-4dea-9749-04f9e2791ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.637752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabf677-a632-4dea-9749-04f9e2791ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.638475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabf677-a632-4dea-9749-04f9e2791ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.638948 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eabf677-a632-4dea-9749-04f9e2791ef0 admin admin] PATH_INFO: `/v3/projects/e33ae45043cb4faf8f602688a70383f3/users/f06c0b8d372c4885950e470e3f3a911e/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.643106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c531d-2f5e-4608-8bd3-32b6b6b031e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a5cc1d2dee4412eb37b794156fdcb83, user_id=24f34c5beaad416188a32cf0860dd93e, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.647508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c531d-2f5e-4608-8bd3-32b6b6b031e1 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.666934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1152/2303] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:48 2022] PUT /identity/v3/projects/2a5cc1d2dee4412eb37b794156fdcb83/users/24f34c5beaad416188a32cf0860dd93e/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:48.668129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabf677-a632-4dea-9749-04f9e2791ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e33ae45043cb4faf8f602688a70383f3, user_id=f06c0b8d372c4885950e470e3f3a911e, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.671127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eabf677-a632-4dea-9749-04f9e2791ef0 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.671896 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314443d3-7ea8-4e17-b97a-65360e4651fb None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.672421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314443d3-7ea8-4e17-b97a-65360e4651fb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.673157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314443d3-7ea8-4e17-b97a-65360e4651fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.691415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1152/2304] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:48 2022] PUT /identity/v3/projects/e33ae45043cb4faf8f602688a70383f3/users/f06c0b8d372c4885950e470e3f3a911e/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:48.699786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-314443d3-7ea8-4e17-b97a-65360e4651fb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:48.700942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-314443d3-7ea8-4e17-b97a-65360e4651fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:48.705615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02421aab-ef0d-4ff9-a9d5-052d60766614 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.706616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02421aab-ef0d-4ff9-a9d5-052d60766614 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.706933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02421aab-ef0d-4ff9-a9d5-052d60766614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.739425 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-02421aab-ef0d-4ff9-a9d5-052d60766614 None None] MFA Rules not processed for user `f06c0b8d372c4885950e470e3f3a911e`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:48.741104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-02421aab-ef0d-4ff9-a9d5-052d60766614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:48.776416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1153/2305] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:48.777525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:48.788047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314443d3-7ea8-4e17-b97a-65360e4651fb None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.790531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-314443d3-7ea8-4e17-b97a-65360e4651fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:48.848970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1153/2306] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:48.852959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeea70e-8d35-421b-9df8-33a65d783644 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.852959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeea70e-8d35-421b-9df8-33a65d783644 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.852959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeea70e-8d35-421b-9df8-33a65d783644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.866794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314443d3-7ea8-4e17-b97a-65360e4651fb None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.872332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77b2f1-6280-4df4-b3e8-5b2dabbf443a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.873001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77b2f1-6280-4df4-b3e8-5b2dabbf443a service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.873282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77b2f1-6280-4df4-b3e8-5b2dabbf443a service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.873939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77b2f1-6280-4df4-b3e8-5b2dabbf443a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.886222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-9eeea70e-8d35-421b-9df8-33a65d783644 None None] MFA Rules not processed for user `f82ce1e8ee7d441e9365de3caf8f6c6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:48.886428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77b2f1-6280-4df4-b3e8-5b2dabbf443a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.887936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9eeea70e-8d35-421b-9df8-33a65d783644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:48.888083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77b2f1-6280-4df4-b3e8-5b2dabbf443a service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.898817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1154/2307] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:06:48.899806 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77b2f1-6280-4df4-b3e8-5b2dabbf443a service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.909795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77b2f1-6280-4df4-b3e8-5b2dabbf443a service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:48.914962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e5e77-049e-4061-8ce9-02ec86d2cb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:48.915598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e5e77-049e-4061-8ce9-02ec86d2cb9a admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:48.915881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e5e77-049e-4061-8ce9-02ec86d2cb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:48.916155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e5e77-049e-4061-8ce9-02ec86d2cb9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:48.916867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e5e77-049e-4061-8ce9-02ec86d2cb9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:48.918868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e5e77-049e-4061-8ce9-02ec86d2cb9a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:48.952441 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1155/2308] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:48.956720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e5e77-049e-4061-8ce9-02ec86d2cb9a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:48.963434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:48.995837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1154/2309] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:49.008042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeea70e-8d35-421b-9df8-33a65d783644 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:49.010606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9eeea70e-8d35-421b-9df8-33a65d783644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:49.060108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e5e77-049e-4061-8ce9-02ec86d2cb9a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:49.066252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f587f3b-b67b-4845-b3f8-c0a7d4315bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:49.067352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f587f3b-b67b-4845-b3f8-c0a7d4315bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:49.067711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f587f3b-b67b-4845-b3f8-c0a7d4315bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:49.067990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f587f3b-b67b-4845-b3f8-c0a7d4315bdc admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:49.069454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f587f3b-b67b-4845-b3f8-c0a7d4315bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:49.072860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f587f3b-b67b-4845-b3f8-c0a7d4315bdc admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:49.083501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeea70e-8d35-421b-9df8-33a65d783644 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:49.106619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1156/2310] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:48 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:49.108970 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f587f3b-b67b-4845-b3f8-c0a7d4315bdc admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:49.135614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f587f3b-b67b-4845-b3f8-c0a7d4315bdc admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:49.141588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a0aef-15da-44c3-841d-078949a8161c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:49.142220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a0aef-15da-44c3-841d-078949a8161c admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:49.142499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a0aef-15da-44c3-841d-078949a8161c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:49.142767 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a0aef-15da-44c3-841d-078949a8161c admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:49.143577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a0aef-15da-44c3-841d-078949a8161c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:49.145243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a0aef-15da-44c3-841d-078949a8161c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:49.153188 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0b4d7-7d09-4770-b1dd-feb18e05e6f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:49.154285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0b4d7-7d09-4770-b1dd-feb18e05e6f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:49.154779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0b4d7-7d09-4770-b1dd-feb18e05e6f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:49.155270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0b4d7-7d09-4770-b1dd-feb18e05e6f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:49.161809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1157/2311] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:49.165470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a0aef-15da-44c3-841d-078949a8161c admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:49.178036 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a0aef-15da-44c3-841d-078949a8161c admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:49.184045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3022b90-1a90-437c-97f1-fa5863c65a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:49.184803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3022b90-1a90-437c-97f1-fa5863c65a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:49.185079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3022b90-1a90-437c-97f1-fa5863c65a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:49.185349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3022b90-1a90-437c-97f1-fa5863c65a45 admin admin] PATH_INFO: `/v3/projects/1aef0323320a47c4871fa8418d5bccd6/users/4510fb71656a4f458cac5c8680c1a7b7/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:49.186464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:49.204960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3022b90-1a90-437c-97f1-fa5863c65a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aef0323320a47c4871fa8418d5bccd6, user_id=4510fb71656a4f458cac5c8680c1a7b7, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:49.207779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3022b90-1a90-437c-97f1-fa5863c65a45 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:49.223508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0b4d7-7d09-4770-b1dd-feb18e05e6f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:49.225740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0b4d7-7d09-4770-b1dd-feb18e05e6f8 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:49.226636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1158/2312] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:49 2022] PUT /identity/v3/projects/1aef0323320a47c4871fa8418d5bccd6/users/4510fb71656a4f458cac5c8680c1a7b7/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:49.230988 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3022b90-1a90-437c-97f1-fa5863c65a45 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:49.237209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1155/2313] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:49.308364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3022b90-1a90-437c-97f1-fa5863c65a45 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:49.317041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970998c-4d18-4a54-952f-ded2e51a3f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:49.317781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970998c-4d18-4a54-952f-ded2e51a3f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:49.318135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970998c-4d18-4a54-952f-ded2e51a3f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:49.318478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970998c-4d18-4a54-952f-ded2e51a3f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:49.319255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970998c-4d18-4a54-952f-ded2e51a3f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:49.321390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970998c-4d18-4a54-952f-ded2e51a3f09 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:49.336265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1159/2314] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:49.341005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0b4d7-7d09-4770-b1dd-feb18e05e6f8 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:49.357277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0b4d7-7d09-4770-b1dd-feb18e05e6f8 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:49.363651 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30ef9a-9c07-44d2-94b4-252667b3af3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:49.364455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30ef9a-9c07-44d2-94b4-252667b3af3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:49.364931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30ef9a-9c07-44d2-94b4-252667b3af3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:49.365210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30ef9a-9c07-44d2-94b4-252667b3af3d admin admin] PATH_INFO: `/v3/projects/1aef0323320a47c4871fa8418d5bccd6/users/4510fb71656a4f458cac5c8680c1a7b7/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:49.368471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30ef9a-9c07-44d2-94b4-252667b3af3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aef0323320a47c4871fa8418d5bccd6, user_id=4510fb71656a4f458cac5c8680c1a7b7, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:49.373714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30ef9a-9c07-44d2-94b4-252667b3af3d admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:49.416896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1156/2315] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:49 2022] PUT /identity/v3/projects/1aef0323320a47c4871fa8418d5bccd6/users/4510fb71656a4f458cac5c8680c1a7b7/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:49.419158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540975c-c958-48a2-b3f3-36ef90f36a8a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:49.419650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540975c-c958-48a2-b3f3-36ef90f36a8a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:49.419740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540975c-c958-48a2-b3f3-36ef90f36a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:49.455824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2540975c-c958-48a2-b3f3-36ef90f36a8a None None] MFA Rules not processed for user `4510fb71656a4f458cac5c8680c1a7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:49.457097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2540975c-c958-48a2-b3f3-36ef90f36a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:49.507754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:49.547003 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1160/2316] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:49.570761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30ef9a-9c07-44d2-94b4-252667b3af3d admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:49.639541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30ef9a-9c07-44d2-94b4-252667b3af3d admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:49.644569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368df797-db21-45ef-a147-ca42e9c8e223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:49.645374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368df797-db21-45ef-a147-ca42e9c8e223 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:49.645821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368df797-db21-45ef-a147-ca42e9c8e223 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:49.646359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368df797-db21-45ef-a147-ca42e9c8e223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:49.659068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368df797-db21-45ef-a147-ca42e9c8e223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:49.660606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368df797-db21-45ef-a147-ca42e9c8e223 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:49.683324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1157/2317] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:50.084578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540975c-c958-48a2-b3f3-36ef90f36a8a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:50.094867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540975c-c958-48a2-b3f3-36ef90f36a8a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:50.100221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7bbc5-b96e-41e3-8f7c-edb5bf3945fa tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4510fb71656a4f458cac5c8680c1a7b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef0323320a47c4871fa8418d5bccd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:50.101218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7bbc5-b96e-41e3-8f7c-edb5bf3945fa tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:50.101343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7bbc5-b96e-41e3-8f7c-edb5bf3945fa tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:50.101543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7bbc5-b96e-41e3-8f7c-edb5bf3945fa tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:50.102280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7bbc5-b96e-41e3-8f7c-edb5bf3945fa tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:50.102789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7bbc5-b96e-41e3-8f7c-edb5bf3945fa tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:50.104493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7bbc5-b96e-41e3-8f7c-edb5bf3945fa tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:50.123980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1161/2318] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:50.125366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368df797-db21-45ef-a147-ca42e9c8e223 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:50.137111 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368df797-db21-45ef-a147-ca42e9c8e223 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:50.142431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af37def-ebc1-4a0d-8b39-20b2d9de7a02 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4510fb71656a4f458cac5c8680c1a7b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef0323320a47c4871fa8418d5bccd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:50.143086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af37def-ebc1-4a0d-8b39-20b2d9de7a02 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:50.143430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af37def-ebc1-4a0d-8b39-20b2d9de7a02 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:50.143714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af37def-ebc1-4a0d-8b39-20b2d9de7a02 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:50.144444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af37def-ebc1-4a0d-8b39-20b2d9de7a02 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:50.146464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af37def-ebc1-4a0d-8b39-20b2d9de7a02 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:50.178317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1158/2319] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:50 2022] POST /identity/v3/projects => generated 354 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:50.182653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7bbc5-b96e-41e3-8f7c-edb5bf3945fa tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:50.195176 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7bbc5-b96e-41e3-8f7c-edb5bf3945fa tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:50.265230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0613269-b057-4095-9376-72089ff8aed4 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4510fb71656a4f458cac5c8680c1a7b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef0323320a47c4871fa8418d5bccd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:50.265944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0613269-b057-4095-9376-72089ff8aed4 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:50.266235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0613269-b057-4095-9376-72089ff8aed4 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:50.266538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0613269-b057-4095-9376-72089ff8aed4 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:50.267236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0613269-b057-4095-9376-72089ff8aed4 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:50.267536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0613269-b057-4095-9376-72089ff8aed4 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:50.269241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0613269-b057-4095-9376-72089ff8aed4 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:50.286217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1162/2320] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:50.331441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af37def-ebc1-4a0d-8b39-20b2d9de7a02 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:50.342732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af37def-ebc1-4a0d-8b39-20b2d9de7a02 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:50.347721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4510fb71656a4f458cac5c8680c1a7b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1aef0323320a47c4871fa8418d5bccd6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:50.348458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:50.348749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:50.349026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] PATH_INFO: `/v3/projects/ebc1dfbf389c4440bb630e8ac454ec58` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:50.350282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ebc1dfbf389c4440bb630e8ac454ec58)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:50.352352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:50.417965 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc1dfbf389c4440bb630e8ac454ec58', 'request_id': 'req-58a83b05-6882-4aaa-9588-c8120dda937f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:50.427089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc1dfbf389c4440bb630e8ac454ec58', 'request_id': 'req-58a83b05-6882-4aaa-9588-c8120dda937f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:50.440520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc1dfbf389c4440bb630e8ac454ec58', 'request_id': 'req-58a83b05-6882-4aaa-9588-c8120dda937f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:50.454699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] The token cache is being invalidate because project ebc1dfbf389c4440bb630e8ac454ec58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:50.455240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:50.456001 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-58a83b05-6882-4aaa-9588-c8120dda937f tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:50.458422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1159/2321] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:50 2022] DELETE /identity/v3/projects/ebc1dfbf389c4440bb630e8ac454ec58 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:50.734116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0613269-b057-4095-9376-72089ff8aed4 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:50.735891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a0613269-b057-4095-9376-72089ff8aed4 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:50.856752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0613269-b057-4095-9376-72089ff8aed4 tempest-VolumeQuotaClassesTest-1462485503 tempest-VolumeQuotaClassesTest-1462485503-project-admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:50.926104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:50.926872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:50.927222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:50.927566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] PATH_INFO: `/v3/users/f06c0b8d372c4885950e470e3f3a911e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:50.929297 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f06c0b8d372c4885950e470e3f3a911e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:50.931368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:50.938258 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:50.983454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f06c0b8d372c4885950e470e3f3a911e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:50.984108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f06c0b8d372c4885950e470e3f3a911e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:50.993575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f06c0b8d372c4885950e470e3f3a911e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.007763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f06c0b8d372c4885950e470e3f3a911e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.015313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f06c0b8d372c4885950e470e3f3a911e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.021949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1163/2322] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:50 2022] DELETE /identity/v3/users/f06c0b8d372c4885950e470e3f3a911e => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:51.026626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bde1b-d151-4a51-9bc6-bf0aadf6e9d4 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:51.026985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bde1b-d151-4a51-9bc6-bf0aadf6e9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:51.027329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774bde1b-d151-4a51-9bc6-bf0aadf6e9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:51.046694 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-774bde1b-d151-4a51-9bc6-bf0aadf6e9d4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:51.047906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-774bde1b-d151-4a51-9bc6-bf0aadf6e9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:51.057634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:51.121550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1160/2323] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:51.131905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:51.133568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:51.196337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dab10e-9540-4d92-8704-366a3a795df0 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:51.200396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759ff15-8537-4185-9560-f2a71d4fddf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:51.201046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759ff15-8537-4185-9560-f2a71d4fddf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:51.201326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759ff15-8537-4185-9560-f2a71d4fddf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:51.201599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759ff15-8537-4185-9560-f2a71d4fddf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:51.212211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759ff15-8537-4185-9560-f2a71d4fddf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:51.213791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759ff15-8537-4185-9560-f2a71d4fddf9 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:51.228392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1164/2324] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:51.542892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bde1b-d151-4a51-9bc6-bf0aadf6e9d4 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:51.545128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-774bde1b-d151-4a51-9bc6-bf0aadf6e9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:51.560186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774bde1b-d151-4a51-9bc6-bf0aadf6e9d4 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:51.567693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:51.568757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:51.569170 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:51.569662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] PATH_INFO: `/v3/projects/e33ae45043cb4faf8f602688a70383f3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:51.572010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e33ae45043cb4faf8f602688a70383f3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:51.574521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:51.633784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e33ae45043cb4faf8f602688a70383f3', 'request_id': 'req-18636344-83ed-46f2-82d6-c0ac01fc7bc4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.640483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e33ae45043cb4faf8f602688a70383f3', 'request_id': 'req-18636344-83ed-46f2-82d6-c0ac01fc7bc4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.650979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e33ae45043cb4faf8f602688a70383f3', 'request_id': 'req-18636344-83ed-46f2-82d6-c0ac01fc7bc4'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.664184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] The token cache is being invalidate because project e33ae45043cb4faf8f602688a70383f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:51.664576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.665229 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.667685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1161/2325] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:51 2022] DELETE /identity/v3/projects/e33ae45043cb4faf8f602688a70383f3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:51.672989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759ff15-8537-4185-9560-f2a71d4fddf9 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:51.683881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759ff15-8537-4185-9560-f2a71d4fddf9 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:51.768452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:51.769157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:51.769440 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:51.769773 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] PATH_INFO: `/v3/users/4510fb71656a4f458cac5c8680c1a7b7` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:51.771047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4510fb71656a4f458cac5c8680c1a7b7)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:51.773565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:51.813846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:51.866729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4510fb71656a4f458cac5c8680c1a7b7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.869251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4510fb71656a4f458cac5c8680c1a7b7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.887494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4510fb71656a4f458cac5c8680c1a7b7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.913581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4510fb71656a4f458cac5c8680c1a7b7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.923842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-f8b68002-c156-4070-b5a5-1439e2b7e0a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4510fb71656a4f458cac5c8680c1a7b7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:51.948798 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1165/2326] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:51 2022] DELETE /identity/v3/users/4510fb71656a4f458cac5c8680c1a7b7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:52.376311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:52.378099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:52.400743 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18636344-83ed-46f2-82d6-c0ac01fc7bc4 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:52.479702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:52.480465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:52.480769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:52.481010 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] PATH_INFO: `/v3/projects/1aef0323320a47c4871fa8418d5bccd6` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:52.482403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aef0323320a47c4871fa8418d5bccd6)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:52.484598 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:52.542351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:52.552333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aef0323320a47c4871fa8418d5bccd6', 'request_id': 'req-3a1041e6-0f52-4b23-b304-bb1b46442517'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:52.560332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aef0323320a47c4871fa8418d5bccd6', 'request_id': 'req-3a1041e6-0f52-4b23-b304-bb1b46442517'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:52.569731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aef0323320a47c4871fa8418d5bccd6', 'request_id': 'req-3a1041e6-0f52-4b23-b304-bb1b46442517'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:52.582115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] The token cache is being invalidate because project 1aef0323320a47c4871fa8418d5bccd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:52.582961 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:52.583795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-3a1041e6-0f52-4b23-b304-bb1b46442517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:52.589845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1162/2327] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:52 2022] DELETE /identity/v3/projects/1aef0323320a47c4871fa8418d5bccd6 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:52.592391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c86eb-ea6e-4096-96a6-8627248812a9 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:52.592740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c86eb-ea6e-4096-96a6-8627248812a9 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:52.593054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c86eb-ea6e-4096-96a6-8627248812a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:52.615600 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9c1c86eb-ea6e-4096-96a6-8627248812a9 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:52.617467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9c1c86eb-ea6e-4096-96a6-8627248812a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:52.703298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1166/2328] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:52.711938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154e07-dc64-4c36-8b66-b763cfdbeb15 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:52.712241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154e07-dc64-4c36-8b66-b763cfdbeb15 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:52.712553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70154e07-dc64-4c36-8b66-b763cfdbeb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:52.744923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-70154e07-dc64-4c36-8b66-b763cfdbeb15 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:52.746200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-70154e07-dc64-4c36-8b66-b763cfdbeb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:52.760601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1163/2329] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:52.762439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c86eb-ea6e-4096-96a6-8627248812a9 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:52.785832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c86eb-ea6e-4096-96a6-8627248812a9 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:52.804113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a11e1-8eb1-4a63-a8d2-d6ad3e765d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:52.804827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a11e1-8eb1-4a63-a8d2-d6ad3e765d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:52.805110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a11e1-8eb1-4a63-a8d2-d6ad3e765d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:52.805388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a11e1-8eb1-4a63-a8d2-d6ad3e765d84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:52.806131 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a11e1-8eb1-4a63-a8d2-d6ad3e765d84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:52.813366 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a11e1-8eb1-4a63-a8d2-d6ad3e765d84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:52.815365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a11e1-8eb1-4a63-a8d2-d6ad3e765d84 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:52.840511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1167/2330] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:52.841484 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70154e07-dc64-4c36-8b66-b763cfdbeb15 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:52.879923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7137b6-5187-403e-ab37-e43437bc4319 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:52.880276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7137b6-5187-403e-ab37-e43437bc4319 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:52.880602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70154e07-dc64-4c36-8b66-b763cfdbeb15 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:52.880938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7137b6-5187-403e-ab37-e43437bc4319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:52.889244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d1623-86c4-4ff6-97b9-e88d260401c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:52.890046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d1623-86c4-4ff6-97b9-e88d260401c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:52.890433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d1623-86c4-4ff6-97b9-e88d260401c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:52.890829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d1623-86c4-4ff6-97b9-e88d260401c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:52.891751 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d1623-86c4-4ff6-97b9-e88d260401c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:52.893968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d1623-86c4-4ff6-97b9-e88d260401c2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:52.919426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-1d7137b6-5187-403e-ab37-e43437bc4319 None None] MFA Rules not processed for user `f82ce1e8ee7d441e9365de3caf8f6c6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:52.920907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1d7137b6-5187-403e-ab37-e43437bc4319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:52.960521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1164/2331] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:52 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:52.963365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d1623-86c4-4ff6-97b9-e88d260401c2 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:53.067300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1168/2332] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:53.103543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d1623-86c4-4ff6-97b9-e88d260401c2 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:53.118253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119eb20-40c1-4f73-b3e5-eb9738dad981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:53.118566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119eb20-40c1-4f73-b3e5-eb9738dad981 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:53.118784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119eb20-40c1-4f73-b3e5-eb9738dad981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:53.119061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119eb20-40c1-4f73-b3e5-eb9738dad981 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:53.119800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119eb20-40c1-4f73-b3e5-eb9738dad981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:53.121843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119eb20-40c1-4f73-b3e5-eb9738dad981 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:53.167601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1165/2333] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:52 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:53.170989 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7137b6-5187-403e-ab37-e43437bc4319 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:53.189831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7137b6-5187-403e-ab37-e43437bc4319 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:53.197291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0b2f5-12a8-45d6-9ca0-9d549bd9be4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:53.198593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0b2f5-12a8-45d6-9ca0-9d549bd9be4d admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:53.200436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0b2f5-12a8-45d6-9ca0-9d549bd9be4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:53.200873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0b2f5-12a8-45d6-9ca0-9d549bd9be4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:53.202340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0b2f5-12a8-45d6-9ca0-9d549bd9be4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:53.204593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0b2f5-12a8-45d6-9ca0-9d549bd9be4d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:53.241820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1169/2334] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:53.245927 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119eb20-40c1-4f73-b3e5-eb9738dad981 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:53.257580 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119eb20-40c1-4f73-b3e5-eb9738dad981 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:53.264237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35caa5c0-b50e-4bae-862e-aa6b82fb973c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:53.265022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caa5c0-b50e-4bae-862e-aa6b82fb973c admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:53.265375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caa5c0-b50e-4bae-862e-aa6b82fb973c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:53.266072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35caa5c0-b50e-4bae-862e-aa6b82fb973c admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:53.266793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35caa5c0-b50e-4bae-862e-aa6b82fb973c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:53.269292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35caa5c0-b50e-4bae-862e-aa6b82fb973c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:53.283151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1166/2335] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:53.290387 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0b2f5-12a8-45d6-9ca0-9d549bd9be4d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:53.304006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0b2f5-12a8-45d6-9ca0-9d549bd9be4d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:53.311669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e69ef-78f0-45af-9cc9-cd25c14ae065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:53.312598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e69ef-78f0-45af-9cc9-cd25c14ae065 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:53.313070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e69ef-78f0-45af-9cc9-cd25c14ae065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:53.313430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e69ef-78f0-45af-9cc9-cd25c14ae065 admin admin] PATH_INFO: `/v3/projects/88e40399726b44bf9ff3af66ec2fa3d6/users/4cf9a6ba14514973a508aa12086c37b8/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:53.345808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e69ef-78f0-45af-9cc9-cd25c14ae065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e40399726b44bf9ff3af66ec2fa3d6, user_id=4cf9a6ba14514973a508aa12086c37b8, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:53.348744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e69ef-78f0-45af-9cc9-cd25c14ae065 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:53.377069 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1170/2336] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:53 2022] PUT /identity/v3/projects/88e40399726b44bf9ff3af66ec2fa3d6/users/4cf9a6ba14514973a508aa12086c37b8/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:53.390119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60466bfc-1232-4aa0-be1a-944410b22211 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:53.390330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60466bfc-1232-4aa0-be1a-944410b22211 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:53.390582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60466bfc-1232-4aa0-be1a-944410b22211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:53.438080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-60466bfc-1232-4aa0-be1a-944410b22211 None None] MFA Rules not processed for user `4cf9a6ba14514973a508aa12086c37b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:53.439942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-60466bfc-1232-4aa0-be1a-944410b22211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:53.568159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1167/2337] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:53.578606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e69ef-78f0-45af-9cc9-cd25c14ae065 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:53.588802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:53.705784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e69ef-78f0-45af-9cc9-cd25c14ae065 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:53.715811 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bb6c0-2bf8-42b6-8f7c-36db47bbd21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:53.716630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bb6c0-2bf8-42b6-8f7c-36db47bbd21b admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:53.716915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bb6c0-2bf8-42b6-8f7c-36db47bbd21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:53.717184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bb6c0-2bf8-42b6-8f7c-36db47bbd21b admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:53.717870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bb6c0-2bf8-42b6-8f7c-36db47bbd21b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:53.724516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bb6c0-2bf8-42b6-8f7c-36db47bbd21b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:53.784847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1171/2338] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:53 2022] POST /identity/v3/projects => generated 389 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:53.790348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60466bfc-1232-4aa0-be1a-944410b22211 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:53.808766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60466bfc-1232-4aa0-be1a-944410b22211 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:53.823957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:53.983210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cf9cc-666e-4ca6-90ea-f2e6dec9e214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:53.984062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf9cc-666e-4ca6-90ea-f2e6dec9e214 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:53.984412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf9cc-666e-4ca6-90ea-f2e6dec9e214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:53.984689 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980cf9cc-666e-4ca6-90ea-f2e6dec9e214 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:53.985388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cf9cc-666e-4ca6-90ea-f2e6dec9e214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:53.990257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980cf9cc-666e-4ca6-90ea-f2e6dec9e214 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:54.033588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf2c3d-1048-4741-86e6-93bae5af1345 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.034966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf2c3d-1048-4741-86e6-93bae5af1345 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.035895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf2c3d-1048-4741-86e6-93bae5af1345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.042818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1168/2339] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:53 2022] POST /identity/v3/users => generated 301 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:54.046629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cf9cc-666e-4ca6-90ea-f2e6dec9e214 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.080620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980cf9cc-666e-4ca6-90ea-f2e6dec9e214 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.087567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44b8cb-0afe-45b2-82f8-ed119bfaa7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.088276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44b8cb-0afe-45b2-82f8-ed119bfaa7eb admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.088712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44b8cb-0afe-45b2-82f8-ed119bfaa7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.089073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44b8cb-0afe-45b2-82f8-ed119bfaa7eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.089880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44b8cb-0afe-45b2-82f8-ed119bfaa7eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:54.091730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44b8cb-0afe-45b2-82f8-ed119bfaa7eb admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:54.103354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-0eaf2c3d-1048-4741-86e6-93bae5af1345 None None] MFA Rules not processed for user `24f34c5beaad416188a32cf0860dd93e`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:54.104710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0eaf2c3d-1048-4741-86e6-93bae5af1345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:54.138192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1169/2340] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:54.142265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44b8cb-0afe-45b2-82f8-ed119bfaa7eb admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.180284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44b8cb-0afe-45b2-82f8-ed119bfaa7eb admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.186183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1e466-c663-4fa3-ba92-6311cb831e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.187011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1e466-c663-4fa3-ba92-6311cb831e23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.187398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1e466-c663-4fa3-ba92-6311cb831e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.187764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc1e466-c663-4fa3-ba92-6311cb831e23 admin admin] PATH_INFO: `/v3/projects/f19b01b8ba7a41ad8ce3f00874162ad9/users/4bb0576f00874f2aa76ceec8b91a8d1f/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.211507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1172/2341] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:54.216081 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf2c3d-1048-4741-86e6-93bae5af1345 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.218495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0eaf2c3d-1048-4741-86e6-93bae5af1345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:54.220642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1e466-c663-4fa3-ba92-6311cb831e23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19b01b8ba7a41ad8ce3f00874162ad9, user_id=4bb0576f00874f2aa76ceec8b91a8d1f, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:54.224437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc1e466-c663-4fa3-ba92-6311cb831e23 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:54.245847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf2c3d-1048-4741-86e6-93bae5af1345 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.248992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1170/2342] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:54 2022] PUT /identity/v3/projects/f19b01b8ba7a41ad8ce3f00874162ad9/users/4bb0576f00874f2aa76ceec8b91a8d1f/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:54.253664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1e466-c663-4fa3-ba92-6311cb831e23 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.338225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe34feb-5992-4152-962b-83ee96ccdda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.338829 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe34feb-5992-4152-962b-83ee96ccdda5 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.339178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe34feb-5992-4152-962b-83ee96ccdda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.339729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe34feb-5992-4152-962b-83ee96ccdda5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.340632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe34feb-5992-4152-962b-83ee96ccdda5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:54.343722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe34feb-5992-4152-962b-83ee96ccdda5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:54.346806 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc1e466-c663-4fa3-ba92-6311cb831e23 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.356894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb5242-e7b1-4b23-8ba2-a5dd3aeb2b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.358086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5242-e7b1-4b23-8ba2-a5dd3aeb2b27 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.358441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5242-e7b1-4b23-8ba2-a5dd3aeb2b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.358784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb5242-e7b1-4b23-8ba2-a5dd3aeb2b27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.359726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb5242-e7b1-4b23-8ba2-a5dd3aeb2b27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:54.363468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb5242-e7b1-4b23-8ba2-a5dd3aeb2b27 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:54.381968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1171/2343] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:54.390912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb5242-e7b1-4b23-8ba2-a5dd3aeb2b27 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.435655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1173/2344] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:54 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:54.436524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb5242-e7b1-4b23-8ba2-a5dd3aeb2b27 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.439630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe34feb-5992-4152-962b-83ee96ccdda5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.446225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:54.540403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc448c9-52aa-4402-8c73-f94670bfecf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.541100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc448c9-52aa-4402-8c73-f94670bfecf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.541386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc448c9-52aa-4402-8c73-f94670bfecf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.541660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc448c9-52aa-4402-8c73-f94670bfecf9 admin admin] PATH_INFO: `/v3/projects/f19b01b8ba7a41ad8ce3f00874162ad9/users/4bb0576f00874f2aa76ceec8b91a8d1f/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.543691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc448c9-52aa-4402-8c73-f94670bfecf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19b01b8ba7a41ad8ce3f00874162ad9, user_id=4bb0576f00874f2aa76ceec8b91a8d1f, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:54.544306 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe34feb-5992-4152-962b-83ee96ccdda5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.546449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc448c9-52aa-4402-8c73-f94670bfecf9 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:54.550474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af73e79-1d9c-4441-8e20-494d160728dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.551497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af73e79-1d9c-4441-8e20-494d160728dc admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.551634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af73e79-1d9c-4441-8e20-494d160728dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.551911 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af73e79-1d9c-4441-8e20-494d160728dc admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.552152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:54.552842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af73e79-1d9c-4441-8e20-494d160728dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:54.554746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af73e79-1d9c-4441-8e20-494d160728dc admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:54.561193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1172/2345] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:54 2022] PUT /identity/v3/projects/f19b01b8ba7a41ad8ce3f00874162ad9/users/4bb0576f00874f2aa76ceec8b91a8d1f/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:54.572925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa843-84c0-4cb7-a1a3-6d107adc743a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.573923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa843-84c0-4cb7-a1a3-6d107adc743a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.574351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29fa843-84c0-4cb7-a1a3-6d107adc743a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.583292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1174/2346] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:54 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:06:54.586864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af73e79-1d9c-4441-8e20-494d160728dc admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.604926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af73e79-1d9c-4441-8e20-494d160728dc admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.620615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b29fa843-84c0-4cb7-a1a3-6d107adc743a None None] MFA Rules not processed for user `4bb0576f00874f2aa76ceec8b91a8d1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:54.622070 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b29fa843-84c0-4cb7-a1a3-6d107adc743a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:54.689550 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59267f91-da2f-4c05-a4b0-3b4e85f2ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.690212 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59267f91-da2f-4c05-a4b0-3b4e85f2ce71 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.690657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59267f91-da2f-4c05-a4b0-3b4e85f2ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.690923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59267f91-da2f-4c05-a4b0-3b4e85f2ce71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.691750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59267f91-da2f-4c05-a4b0-3b4e85f2ce71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:54.693545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59267f91-da2f-4c05-a4b0-3b4e85f2ce71 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:54.709255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1175/2347] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:54.710426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59267f91-da2f-4c05-a4b0-3b4e85f2ce71 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.728489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1173/2348] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:54.732698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59267f91-da2f-4c05-a4b0-3b4e85f2ce71 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.739038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a4bb6-e6d7-4fed-812e-e25a330677d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.740110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a4bb6-e6d7-4fed-812e-e25a330677d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.740508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a4bb6-e6d7-4fed-812e-e25a330677d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.740890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a4bb6-e6d7-4fed-812e-e25a330677d7 admin admin] PATH_INFO: `/v3/projects/8b998a586ac34bff992e7d7f49d0bf18/users/7b401a69e1bb4535af245ca77f59717f/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.748019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fa843-84c0-4cb7-a1a3-6d107adc743a None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.749767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b29fa843-84c0-4cb7-a1a3-6d107adc743a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:54.786161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a4bb6-e6d7-4fed-812e-e25a330677d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b998a586ac34bff992e7d7f49d0bf18, user_id=7b401a69e1bb4535af245ca77f59717f, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:54.788884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031a4bb6-e6d7-4fed-812e-e25a330677d7 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:54.832502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1176/2349] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:54 2022] PUT /identity/v3/projects/8b998a586ac34bff992e7d7f49d0bf18/users/7b401a69e1bb4535af245ca77f59717f/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:54.835835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a4bb6-e6d7-4fed-812e-e25a330677d7 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.856199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29fa843-84c0-4cb7-a1a3-6d107adc743a None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.929864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7542f-ff1c-4e17-b9e3-93c94f879335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.931005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7542f-ff1c-4e17-b9e3-93c94f879335 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.931377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7542f-ff1c-4e17-b9e3-93c94f879335 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.931948 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7542f-ff1c-4e17-b9e3-93c94f879335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.935817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a4bb6-e6d7-4fed-812e-e25a330677d7 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.940990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d5e7f-6ddc-4218-9e9f-b91eb62d9788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.941643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d5e7f-6ddc-4218-9e9f-b91eb62d9788 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.941908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d5e7f-6ddc-4218-9e9f-b91eb62d9788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.942183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d5e7f-6ddc-4218-9e9f-b91eb62d9788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.942890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d5e7f-6ddc-4218-9e9f-b91eb62d9788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:54.944608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d5e7f-6ddc-4218-9e9f-b91eb62d9788 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:54.957467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1177/2350] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:54.962250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d5e7f-6ddc-4218-9e9f-b91eb62d9788 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:54.968718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:54.983209 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d5e7f-6ddc-4218-9e9f-b91eb62d9788 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:54.991851 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773dbb1-5c8a-4d76-a722-194aec0b80e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:54.993383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773dbb1-5c8a-4d76-a722-194aec0b80e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:54.993804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773dbb1-5c8a-4d76-a722-194aec0b80e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:54.994369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773dbb1-5c8a-4d76-a722-194aec0b80e8 admin admin] PATH_INFO: `/v3/projects/8b998a586ac34bff992e7d7f49d0bf18/users/7b401a69e1bb4535af245ca77f59717f/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:54.997075 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773dbb1-5c8a-4d76-a722-194aec0b80e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b998a586ac34bff992e7d7f49d0bf18, user_id=7b401a69e1bb4535af245ca77f59717f, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:55.001153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773dbb1-5c8a-4d76-a722-194aec0b80e8 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:55.011991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:55.048571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1178/2351] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:54 2022] PUT /identity/v3/projects/8b998a586ac34bff992e7d7f49d0bf18/users/7b401a69e1bb4535af245ca77f59717f/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:06:55.060957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7542f-ff1c-4e17-b9e3-93c94f879335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:55.062522 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7542f-ff1c-4e17-b9e3-93c94f879335 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:55.073058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1174/2352] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:55.302147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773dbb1-5c8a-4d76-a722-194aec0b80e8 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:55.412852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773dbb1-5c8a-4d76-a722-194aec0b80e8 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:55.418231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:55.419074 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:55.419353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:55.419624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] PATH_INFO: `/v3/users/4cf9a6ba14514973a508aa12086c37b8` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:55.421087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf9a6ba14514973a508aa12086c37b8)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:55.423211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:55.516852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf9a6ba14514973a508aa12086c37b8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:55.517586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf9a6ba14514973a508aa12086c37b8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:55.532783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf9a6ba14514973a508aa12086c37b8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:55.552937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf9a6ba14514973a508aa12086c37b8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:55.563434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf9a6ba14514973a508aa12086c37b8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:55.575586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1179/2353] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:55 2022] DELETE /identity/v3/users/4cf9a6ba14514973a508aa12086c37b8 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:55.581274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8340658-ed2a-4269-8986-b92e736da601 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:55.581822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8340658-ed2a-4269-8986-b92e736da601 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:55.581822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8340658-ed2a-4269-8986-b92e736da601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:55.608686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d8340658-ed2a-4269-8986-b92e736da601 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:55.610090 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d8340658-ed2a-4269-8986-b92e736da601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:55.684782 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1175/2354] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:55.715679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:55.718396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:55.797536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c117d-bd5f-486b-8600-f747918f0aaf admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:55.804517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b62cf-2c75-4ff9-93cd-cd774db88507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:55.805836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b62cf-2c75-4ff9-93cd-cd774db88507 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:55.806135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b62cf-2c75-4ff9-93cd-cd774db88507 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:55.806418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b62cf-2c75-4ff9-93cd-cd774db88507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:55.819786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b62cf-2c75-4ff9-93cd-cd774db88507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:55.821329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b62cf-2c75-4ff9-93cd-cd774db88507 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:55.837503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1180/2355] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:06:56.647348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8340658-ed2a-4269-8986-b92e736da601 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:56.650500 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d8340658-ed2a-4269-8986-b92e736da601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:56.678319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8340658-ed2a-4269-8986-b92e736da601 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:56.685727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:56.686771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:56.687904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:56.690044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] PATH_INFO: `/v3/projects/88e40399726b44bf9ff3af66ec2fa3d6` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:56.692179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e40399726b44bf9ff3af66ec2fa3d6)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:56.695836 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:56.769009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e40399726b44bf9ff3af66ec2fa3d6', 'request_id': 'req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:56.783067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e40399726b44bf9ff3af66ec2fa3d6', 'request_id': 'req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:56.799532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e40399726b44bf9ff3af66ec2fa3d6', 'request_id': 'req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:56.824563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] The token cache is being invalidate because project 88e40399726b44bf9ff3af66ec2fa3d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:56.825290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:56.827219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:56.830060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1176/2356] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:56 2022] DELETE /identity/v3/projects/88e40399726b44bf9ff3af66ec2fa3d6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:56.834328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b62cf-2c75-4ff9-93cd-cd774db88507 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:56.845869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b62cf-2c75-4ff9-93cd-cd774db88507 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:56.867425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:56.955894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:56.956866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:56.958856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:56.958856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] PATH_INFO: `/v3/users/4bb0576f00874f2aa76ceec8b91a8d1f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:56.959258 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bb0576f00874f2aa76ceec8b91a8d1f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:56.963428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:57.040812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bb0576f00874f2aa76ceec8b91a8d1f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:57.042234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bb0576f00874f2aa76ceec8b91a8d1f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:57.055833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bb0576f00874f2aa76ceec8b91a8d1f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:57.079136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bb0576f00874f2aa76ceec8b91a8d1f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:57.096512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b2267c09-b84f-4221-9222-53667e1a529b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bb0576f00874f2aa76ceec8b91a8d1f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:57.106996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1181/2357] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:06:56 2022] DELETE /identity/v3/users/4bb0576f00874f2aa76ceec8b91a8d1f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:58.710479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:58.712510 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:58.738984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73f1c3e-2a15-49e1-bbd6-fdf8e6ed1fc1 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:58.906646 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:58.907369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:58.907653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:58.908081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] PATH_INFO: `/v3/projects/f19b01b8ba7a41ad8ce3f00874162ad9` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:58.909524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f19b01b8ba7a41ad8ce3f00874162ad9)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:58.912104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:58.933388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:59.016693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f19b01b8ba7a41ad8ce3f00874162ad9', 'request_id': 'req-a967b187-0669-41d5-96bd-7403ce5a9b27'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:59.032953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f19b01b8ba7a41ad8ce3f00874162ad9', 'request_id': 'req-a967b187-0669-41d5-96bd-7403ce5a9b27'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:59.050274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f19b01b8ba7a41ad8ce3f00874162ad9', 'request_id': 'req-a967b187-0669-41d5-96bd-7403ce5a9b27'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:59.069450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] The token cache is being invalidate because project f19b01b8ba7a41ad8ce3f00874162ad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:06:59.069741 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:59.070502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a967b187-0669-41d5-96bd-7403ce5a9b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:06:59.073042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1177/2358] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:06:58 2022] DELETE /identity/v3/projects/f19b01b8ba7a41ad8ce3f00874162ad9 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:59.077857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237faee-def2-42e9-a143-569397b543e7 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:59.079136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237faee-def2-42e9-a143-569397b543e7 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:59.079602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237faee-def2-42e9-a143-569397b543e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:59.108182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-1237faee-def2-42e9-a143-569397b543e7 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:59.110292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1237faee-def2-42e9-a143-569397b543e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:59.232904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1182/2359] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:06:59.240852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00587cd9-e4a5-47d5-9a96-d0057f0c536a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:59.241518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00587cd9-e4a5-47d5-9a96-d0057f0c536a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:59.242003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00587cd9-e4a5-47d5-9a96-d0057f0c536a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:59.303529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-00587cd9-e4a5-47d5-9a96-d0057f0c536a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:06:59.305275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-00587cd9-e4a5-47d5-9a96-d0057f0c536a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:06:59.332150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1178/2360] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:06:59.336569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237faee-def2-42e9-a143-569397b543e7 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:59.352816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237faee-def2-42e9-a143-569397b543e7 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:59.377781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57c00f-5db7-4762-bdac-291cacf5a323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:59.378699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57c00f-5db7-4762-bdac-291cacf5a323 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:59.379050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57c00f-5db7-4762-bdac-291cacf5a323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:59.379419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c57c00f-5db7-4762-bdac-291cacf5a323 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:59.380144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57c00f-5db7-4762-bdac-291cacf5a323 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:06:59.380483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57c00f-5db7-4762-bdac-291cacf5a323 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:59.382186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c57c00f-5db7-4762-bdac-291cacf5a323 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:59.408655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1183/2361] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:06:59.413153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00587cd9-e4a5-47d5-9a96-d0057f0c536a None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:59.438327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00587cd9-e4a5-47d5-9a96-d0057f0c536a None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:59.445396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfb99-7973-4893-91ac-27883e678081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:59.446061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfb99-7973-4893-91ac-27883e678081 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:59.446341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfb99-7973-4893-91ac-27883e678081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:59.446623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfb99-7973-4893-91ac-27883e678081 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:59.447531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bfb99-7973-4893-91ac-27883e678081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:59.451683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bfb99-7973-4893-91ac-27883e678081 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:59.533816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1179/2362] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:06:59 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:59.539958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57c00f-5db7-4762-bdac-291cacf5a323 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:59.558608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c57c00f-5db7-4762-bdac-291cacf5a323 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:59.575532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:06:59.648145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341bb0b-4865-42fe-9b14-7ea01b388b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:59.649035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341bb0b-4865-42fe-9b14-7ea01b388b8c admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:59.650262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341bb0b-4865-42fe-9b14-7ea01b388b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:59.650643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341bb0b-4865-42fe-9b14-7ea01b388b8c admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:59.651460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341bb0b-4865-42fe-9b14-7ea01b388b8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:59.657214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341bb0b-4865-42fe-9b14-7ea01b388b8c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:59.715699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1184/2363] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:06:59 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:06:59.719994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfb99-7973-4893-91ac-27883e678081 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:59.733155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfb99-7973-4893-91ac-27883e678081 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:59.741355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f07e1-7c28-48ba-9642-f66c49dc7fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:59.745203 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f07e1-7c28-48ba-9642-f66c49dc7fad admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:59.746291 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f07e1-7c28-48ba-9642-f66c49dc7fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:59.746777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f07e1-7c28-48ba-9642-f66c49dc7fad admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:59.749087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f07e1-7c28-48ba-9642-f66c49dc7fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:59.751342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f07e1-7c28-48ba-9642-f66c49dc7fad admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:59.780330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1180/2364] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:06:59.784594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341bb0b-4865-42fe-9b14-7ea01b388b8c admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:59.798262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341bb0b-4865-42fe-9b14-7ea01b388b8c admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:59.805052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458b9ae-f906-4e92-9067-3977eb4f8937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:59.806121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458b9ae-f906-4e92-9067-3977eb4f8937 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:59.806650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458b9ae-f906-4e92-9067-3977eb4f8937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:59.807200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458b9ae-f906-4e92-9067-3977eb4f8937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:59.810245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458b9ae-f906-4e92-9067-3977eb4f8937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:59.812633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458b9ae-f906-4e92-9067-3977eb4f8937 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:59.840503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1185/2365] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:06:59.841509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f07e1-7c28-48ba-9642-f66c49dc7fad admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:06:59.856185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f07e1-7c28-48ba-9642-f66c49dc7fad admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:06:59.897051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35d933-1eb0-4be0-825c-770293ea4471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:06:59.898115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35d933-1eb0-4be0-825c-770293ea4471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:59.898793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35d933-1eb0-4be0-825c-770293ea4471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:59.899435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35d933-1eb0-4be0-825c-770293ea4471 admin admin] PATH_INFO: `/v3/projects/c8f3eb844e23466185b5e2dea54a50b6/users/e4d3f9d6e2c5496f9e8904b833159359/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:06:59.925005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35d933-1eb0-4be0-825c-770293ea4471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f3eb844e23466185b5e2dea54a50b6, user_id=e4d3f9d6e2c5496f9e8904b833159359, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:06:59.928108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35d933-1eb0-4be0-825c-770293ea4471 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:06:59.973866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1181/2366] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:06:59 2022] PUT /identity/v3/projects/c8f3eb844e23466185b5e2dea54a50b6/users/e4d3f9d6e2c5496f9e8904b833159359/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:06:59.985676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbd151-51f3-4a21-b9cd-d7010d1bab71 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:06:59.986038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbd151-51f3-4a21-b9cd-d7010d1bab71 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:06:59.986239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbd151-51f3-4a21-b9cd-d7010d1bab71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:00.037897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-78dbd151-51f3-4a21-b9cd-d7010d1bab71 None None] MFA Rules not processed for user `e4d3f9d6e2c5496f9e8904b833159359`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:00.046418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-78dbd151-51f3-4a21-b9cd-d7010d1bab71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:00.186049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1186/2367] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:00.195453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35d933-1eb0-4be0-825c-770293ea4471 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:00.197292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3f35d933-1eb0-4be0-825c-770293ea4471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:00.333363 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35d933-1eb0-4be0-825c-770293ea4471 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:00.338278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcd8df-8f50-4083-b69f-44bf977c5121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:00.339095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd8df-8f50-4083-b69f-44bf977c5121 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:00.339381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd8df-8f50-4083-b69f-44bf977c5121 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:00.339655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd8df-8f50-4083-b69f-44bf977c5121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:00.352807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcd8df-8f50-4083-b69f-44bf977c5121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:00.354605 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcd8df-8f50-4083-b69f-44bf977c5121 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:00.370475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1182/2368] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:07:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:07:01.554689 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60c448-4e1b-47ed-9c38-2e7dce9a0d97 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:01.555215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60c448-4e1b-47ed-9c38-2e7dce9a0d97 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:01.555504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60c448-4e1b-47ed-9c38-2e7dce9a0d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:01.584943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8b60c448-4e1b-47ed-9c38-2e7dce9a0d97 None None] MFA Rules not processed for user `7b401a69e1bb4535af245ca77f59717f`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:01.586204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8b60c448-4e1b-47ed-9c38-2e7dce9a0d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:01.665482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1187/2369] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:01.679217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcd8df-8f50-4083-b69f-44bf977c5121 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:01.680844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-23bcd8df-8f50-4083-b69f-44bf977c5121 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:01.727497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:01.767893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcd8df-8f50-4083-b69f-44bf977c5121 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:01.774135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013abc5-4f1f-4871-90f9-7e677def4492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:01.774947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013abc5-4f1f-4871-90f9-7e677def4492 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:01.775234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013abc5-4f1f-4871-90f9-7e677def4492 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:01.775536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013abc5-4f1f-4871-90f9-7e677def4492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:01.777684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8013abc5-4f1f-4871-90f9-7e677def4492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:01.850194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013abc5-4f1f-4871-90f9-7e677def4492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:01.852060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013abc5-4f1f-4871-90f9-7e677def4492 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:01.864389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1183/2370] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:07:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:07:03.223841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607f942-cdba-4763-a9af-2284aaf07309 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:03.224409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607f942-cdba-4763-a9af-2284aaf07309 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:03.224513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607f942-cdba-4763-a9af-2284aaf07309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:03.245156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-1607f942-cdba-4763-a9af-2284aaf07309 None None] MFA Rules not processed for user `24f34c5beaad416188a32cf0860dd93e`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:03.246309 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1607f942-cdba-4763-a9af-2284aaf07309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:03.254517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1188/2371] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:09.478419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013abc5-4f1f-4871-90f9-7e677def4492 service nova] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:09.567509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013abc5-4f1f-4871-90f9-7e677def4492 service nova] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:09.575884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:09.576635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:09.576946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:09.577368 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] PATH_INFO: `/v3/users/e4d3f9d6e2c5496f9e8904b833159359` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:09.578674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d3f9d6e2c5496f9e8904b833159359)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:09.581372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:09.661788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d3f9d6e2c5496f9e8904b833159359'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:09.677253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d3f9d6e2c5496f9e8904b833159359'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:09.684430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d3f9d6e2c5496f9e8904b833159359'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:09.685217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d3f9d6e2c5496f9e8904b833159359'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:09.685813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d3f9d6e2c5496f9e8904b833159359'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:09.689641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:09.704180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1184/2372] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:07:09 2022] DELETE /identity/v3/users/e4d3f9d6e2c5496f9e8904b833159359 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:07:09.708934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63f49b-e682-4a66-a82d-3e8943993aec None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:09.709766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63f49b-e682-4a66-a82d-3e8943993aec None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:09.710042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63f49b-e682-4a66-a82d-3e8943993aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:09.718726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:09.731528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3d63f49b-e682-4a66-a82d-3e8943993aec None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:09.732797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3d63f49b-e682-4a66-a82d-3e8943993aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:09.800014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1189/2373] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:09.812617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:09.814463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:09.888339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27a052-83b5-4d86-8d34-3725cd3d957c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:09.892268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0860906-4757-4dce-9bd5-ec81f966bcdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:09.892960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0860906-4757-4dce-9bd5-ec81f966bcdd service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:09.893271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0860906-4757-4dce-9bd5-ec81f966bcdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:09.893604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0860906-4757-4dce-9bd5-ec81f966bcdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:09.904998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0860906-4757-4dce-9bd5-ec81f966bcdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:09.906518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0860906-4757-4dce-9bd5-ec81f966bcdd service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:09.916764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1185/2374] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:07:10.329221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63f49b-e682-4a66-a82d-3e8943993aec None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:10.331017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3d63f49b-e682-4a66-a82d-3e8943993aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:10.347452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63f49b-e682-4a66-a82d-3e8943993aec None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:10.354964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:10.355907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:10.356452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:10.357203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] PATH_INFO: `/v3/projects/c8f3eb844e23466185b5e2dea54a50b6` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:10.358625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f3eb844e23466185b5e2dea54a50b6)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:10.363481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:10.426441 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f3eb844e23466185b5e2dea54a50b6', 'request_id': 'req-ef4a9b07-f051-4fa3-831d-6bd9a631c669'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:10.435570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f3eb844e23466185b5e2dea54a50b6', 'request_id': 'req-ef4a9b07-f051-4fa3-831d-6bd9a631c669'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:10.448831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f3eb844e23466185b5e2dea54a50b6', 'request_id': 'req-ef4a9b07-f051-4fa3-831d-6bd9a631c669'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:10.463582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] The token cache is being invalidate because project c8f3eb844e23466185b5e2dea54a50b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:07:10.464369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:10.465054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-ef4a9b07-f051-4fa3-831d-6bd9a631c669 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:10.466853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1190/2375] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:07:10 2022] DELETE /identity/v3/projects/c8f3eb844e23466185b5e2dea54a50b6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:07:10.473009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575c520-350c-4de4-9be2-ebd654068c3a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:10.473297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575c520-350c-4de4-9be2-ebd654068c3a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:10.473571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575c520-350c-4de4-9be2-ebd654068c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:10.494939 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3575c520-350c-4de4-9be2-ebd654068c3a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:10.496563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3575c520-350c-4de4-9be2-ebd654068c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:10.594851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1186/2376] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:10.603855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622a587-bebe-4ae5-986a-877e3a72eee6 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:10.604089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622a587-bebe-4ae5-986a-877e3a72eee6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:10.604437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622a587-bebe-4ae5-986a-877e3a72eee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:10.625954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4622a587-bebe-4ae5-986a-877e3a72eee6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:10.627201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4622a587-bebe-4ae5-986a-877e3a72eee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:10.636492 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1191/2377] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:10.642040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575c520-350c-4de4-9be2-ebd654068c3a None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:10.655799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575c520-350c-4de4-9be2-ebd654068c3a None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:10.660702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e634ad0-22e8-41bb-be95-42d514e0317b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:10.661353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634ad0-22e8-41bb-be95-42d514e0317b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:10.661626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634ad0-22e8-41bb-be95-42d514e0317b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:10.661897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e634ad0-22e8-41bb-be95-42d514e0317b admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:10.662612 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e634ad0-22e8-41bb-be95-42d514e0317b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:07:10.662911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e634ad0-22e8-41bb-be95-42d514e0317b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:10.664696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e634ad0-22e8-41bb-be95-42d514e0317b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:10.682260 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1187/2378] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:07:10.686805 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622a587-bebe-4ae5-986a-877e3a72eee6 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:10.696762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622a587-bebe-4ae5-986a-877e3a72eee6 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:10.703307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a394-99d6-4089-91dc-65bf0f121a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:10.703980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a394-99d6-4089-91dc-65bf0f121a05 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:10.704261 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a394-99d6-4089-91dc-65bf0f121a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:10.704575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0a394-99d6-4089-91dc-65bf0f121a05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:10.705282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0a394-99d6-4089-91dc-65bf0f121a05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:10.707219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0a394-99d6-4089-91dc-65bf0f121a05 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:10.755985 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1192/2379] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:07:10 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:07:10.760754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e634ad0-22e8-41bb-be95-42d514e0317b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:10.771258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e634ad0-22e8-41bb-be95-42d514e0317b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:10.830579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb53f3-ffe4-4419-927f-3081b29c8e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:10.831220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb53f3-ffe4-4419-927f-3081b29c8e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:10.831500 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb53f3-ffe4-4419-927f-3081b29c8e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:10.831749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb53f3-ffe4-4419-927f-3081b29c8e2a admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:10.832549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb53f3-ffe4-4419-927f-3081b29c8e2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:10.834499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb53f3-ffe4-4419-927f-3081b29c8e2a admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:10.859068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1188/2380] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:07:10 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:07:10.863504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a394-99d6-4089-91dc-65bf0f121a05 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:10.875608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0a394-99d6-4089-91dc-65bf0f121a05 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:10.880661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861606c-0653-4a4d-92c2-2f4b9a943116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:10.881294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861606c-0653-4a4d-92c2-2f4b9a943116 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:10.881589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861606c-0653-4a4d-92c2-2f4b9a943116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:10.881866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861606c-0653-4a4d-92c2-2f4b9a943116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:10.882645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861606c-0653-4a4d-92c2-2f4b9a943116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:10.884317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861606c-0653-4a4d-92c2-2f4b9a943116 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:10.895738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1193/2381] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:07:10.899925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb53f3-ffe4-4419-927f-3081b29c8e2a admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:10.910627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb53f3-ffe4-4419-927f-3081b29c8e2a admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:10.916205 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b0dea-31bf-4507-ad05-8e74d9bdf903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:10.916889 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b0dea-31bf-4507-ad05-8e74d9bdf903 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:10.917169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b0dea-31bf-4507-ad05-8e74d9bdf903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:10.917449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b0dea-31bf-4507-ad05-8e74d9bdf903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:10.918163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b0dea-31bf-4507-ad05-8e74d9bdf903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:10.920032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b0dea-31bf-4507-ad05-8e74d9bdf903 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:10.932503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1189/2382] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:07:10.936965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861606c-0653-4a4d-92c2-2f4b9a943116 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:10.947645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861606c-0653-4a4d-92c2-2f4b9a943116 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:10.952620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfa395-1cad-4f6c-bfe5-85c9f223c803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:10.953286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfa395-1cad-4f6c-bfe5-85c9f223c803 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:10.953563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfa395-1cad-4f6c-bfe5-85c9f223c803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:10.953936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfa395-1cad-4f6c-bfe5-85c9f223c803 admin admin] PATH_INFO: `/v3/projects/e03ce641f1f74868b0363f0edfd30e7d/users/49c4cc6b50a54e5184da195012748c3c/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:10.969814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bfa395-1cad-4f6c-bfe5-85c9f223c803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03ce641f1f74868b0363f0edfd30e7d, user_id=49c4cc6b50a54e5184da195012748c3c, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:10.972538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bfa395-1cad-4f6c-bfe5-85c9f223c803 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:10.988387 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1194/2383] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:07:10 2022] PUT /identity/v3/projects/e03ce641f1f74868b0363f0edfd30e7d/users/49c4cc6b50a54e5184da195012748c3c/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:07:11.000415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8eb0e1-d338-429e-a671-0d0f84895e1f None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:11.000696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8eb0e1-d338-429e-a671-0d0f84895e1f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:11.000970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8eb0e1-d338-429e-a671-0d0f84895e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:11.008813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:11.032935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-6c8eb0e1-d338-429e-a671-0d0f84895e1f None None] MFA Rules not processed for user `49c4cc6b50a54e5184da195012748c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:11.034675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6c8eb0e1-d338-429e-a671-0d0f84895e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:11.117022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1190/2384] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:10 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:07:15.294391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4912728-bafc-4720-a50f-8da7a30e9f4e None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:15.298886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4912728-bafc-4720-a50f-8da7a30e9f4e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:15.298886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4912728-bafc-4720-a50f-8da7a30e9f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:15.318319 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c4912728-bafc-4720-a50f-8da7a30e9f4e None None] MFA Rules not processed for user `f82ce1e8ee7d441e9365de3caf8f6c6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:15.319845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c4912728-bafc-4720-a50f-8da7a30e9f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:15.346296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:15.410440 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1195/2385] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:15.922298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8eb0e1-d338-429e-a671-0d0f84895e1f None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:15.922849 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6c8eb0e1-d338-429e-a671-0d0f84895e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:15.994922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8eb0e1-d338-429e-a671-0d0f84895e1f None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:15.999045 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47367470-e0e9-4ce4-95ae-ee6d73d1ee5e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:15.999681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47367470-e0e9-4ce4-95ae-ee6d73d1ee5e service glance] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:15.999959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47367470-e0e9-4ce4-95ae-ee6d73d1ee5e service glance] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:16.000230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47367470-e0e9-4ce4-95ae-ee6d73d1ee5e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:16.012235 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47367470-e0e9-4ce4-95ae-ee6d73d1ee5e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:16.016166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47367470-e0e9-4ce4-95ae-ee6d73d1ee5e service glance] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:16.024781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1191/2386] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:07:18.476028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805f1da-49b5-4470-a4c4-d12e798dd48f None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:18.476627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805f1da-49b5-4470-a4c4-d12e798dd48f None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:18.476765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805f1da-49b5-4470-a4c4-d12e798dd48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:18.505328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4805f1da-49b5-4470-a4c4-d12e798dd48f None None] MFA Rules not processed for user `fc8a5fa04b6141329778221169afdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:18.506823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4805f1da-49b5-4470-a4c4-d12e798dd48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:18.579285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1196/2387] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:26.019990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=57 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:28.575783 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=39 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:34.360077 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47367470-e0e9-4ce4-95ae-ee6d73d1ee5e service glance] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:34.361911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-47367470-e0e9-4ce4-95ae-ee6d73d1ee5e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:34.440617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47367470-e0e9-4ce4-95ae-ee6d73d1ee5e service glance] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:34.446972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:34.447650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:34.447925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:34.448195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] PATH_INFO: `/v3/users/709c635becce4f9aae0eeb81ccace346` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:34.449441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=709c635becce4f9aae0eeb81ccace346)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:34.451475 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:34.506532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '709c635becce4f9aae0eeb81ccace346'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:34.519768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '709c635becce4f9aae0eeb81ccace346'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:34.530493 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '709c635becce4f9aae0eeb81ccace346'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:34.531310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '709c635becce4f9aae0eeb81ccace346'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:34.531955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '709c635becce4f9aae0eeb81ccace346'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:34.546684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1192/2388] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:07:34 2022] DELETE /identity/v3/users/709c635becce4f9aae0eeb81ccace346 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:07:34.551808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9d276-deb0-4ec4-a3c3-d3dd0bbe0425 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:34.552254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9d276-deb0-4ec4-a3c3-d3dd0bbe0425 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:34.552500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9d276-deb0-4ec4-a3c3-d3dd0bbe0425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:34.575938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d4b9d276-deb0-4ec4-a3c3-d3dd0bbe0425 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:34.577271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d4b9d276-deb0-4ec4-a3c3-d3dd0bbe0425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:34.649342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1197/2389] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:34.659623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:34.661407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:34.735372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60851b11-ba3e-4ca3-9588-442abef668aa admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:34.739883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41513a1-010b-41c0-bc2a-f77c9cf8a8dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:34.740537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41513a1-010b-41c0-bc2a-f77c9cf8a8dd service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:34.740819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41513a1-010b-41c0-bc2a-f77c9cf8a8dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:34.741110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41513a1-010b-41c0-bc2a-f77c9cf8a8dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:34.752198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41513a1-010b-41c0-bc2a-f77c9cf8a8dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:34.754262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41513a1-010b-41c0-bc2a-f77c9cf8a8dd service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:34.764932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1193/2390] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:07:35.118082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9d276-deb0-4ec4-a3c3-d3dd0bbe0425 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:35.119629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d4b9d276-deb0-4ec4-a3c3-d3dd0bbe0425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:35.134944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9d276-deb0-4ec4-a3c3-d3dd0bbe0425 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:35.140937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:35.141825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:35.142347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:35.143004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] PATH_INFO: `/v3/projects/e263806394b64815ac8677c51a8b893a` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:35.145407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e263806394b64815ac8677c51a8b893a)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:35.148651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:35.211871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e263806394b64815ac8677c51a8b893a', 'request_id': 'req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:35.218370 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e263806394b64815ac8677c51a8b893a', 'request_id': 'req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:35.232395 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e263806394b64815ac8677c51a8b893a', 'request_id': 'req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:35.250542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] The token cache is being invalidate because project e263806394b64815ac8677c51a8b893a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:07:35.250830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:35.251503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-7f7e3df5-f1e1-485a-a1ec-a6610eb04cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:35.256743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1198/2391] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:07:35 2022] DELETE /identity/v3/projects/e263806394b64815ac8677c51a8b893a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:07:35.258684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc72a2e-37a3-4a31-b1fd-c43b7e434d24 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:35.258884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc72a2e-37a3-4a31-b1fd-c43b7e434d24 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:35.259309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc72a2e-37a3-4a31-b1fd-c43b7e434d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:35.287624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-4dc72a2e-37a3-4a31-b1fd-c43b7e434d24 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:35.289035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-4dc72a2e-37a3-4a31-b1fd-c43b7e434d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:35.318087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:35.372828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1194/2392] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:35.374573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151435c-fd86-47a7-8296-98a8788d7528 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:35.374861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151435c-fd86-47a7-8296-98a8788d7528 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:35.375134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7151435c-fd86-47a7-8296-98a8788d7528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:35.401363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7151435c-fd86-47a7-8296-98a8788d7528 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:35.403765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7151435c-fd86-47a7-8296-98a8788d7528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:35.412979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1199/2393] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:07:35.417968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc72a2e-37a3-4a31-b1fd-c43b7e434d24 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:35.428416 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc72a2e-37a3-4a31-b1fd-c43b7e434d24 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:35.435273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c728b-32f2-4c0f-9d67-a54ea6cee1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:35.436515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c728b-32f2-4c0f-9d67-a54ea6cee1e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:35.436789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c728b-32f2-4c0f-9d67-a54ea6cee1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:35.437054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c728b-32f2-4c0f-9d67-a54ea6cee1e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:35.437768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c728b-32f2-4c0f-9d67-a54ea6cee1e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:07:35.438068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c728b-32f2-4c0f-9d67-a54ea6cee1e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:35.439720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c728b-32f2-4c0f-9d67-a54ea6cee1e3 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:35.456063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1195/2394] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:07:35.457342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151435c-fd86-47a7-8296-98a8788d7528 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:35.469943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7151435c-fd86-47a7-8296-98a8788d7528 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:35.476187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee6184-e69a-4919-8cf5-3e33a98d99ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:35.476855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6184-e69a-4919-8cf5-3e33a98d99ef admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:35.477132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6184-e69a-4919-8cf5-3e33a98d99ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:35.477401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6184-e69a-4919-8cf5-3e33a98d99ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:35.478241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee6184-e69a-4919-8cf5-3e33a98d99ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:35.480149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee6184-e69a-4919-8cf5-3e33a98d99ef admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:35.509796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1200/2395] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:07:35 2022] POST /identity/v3/projects => generated 397 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:07:35.514245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c728b-32f2-4c0f-9d67-a54ea6cee1e3 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:35.525431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c728b-32f2-4c0f-9d67-a54ea6cee1e3 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:35.586565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552c544-95db-436b-b82a-106a087ce332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:35.587223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c544-95db-436b-b82a-106a087ce332 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:35.587492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c544-95db-436b-b82a-106a087ce332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:35.587757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552c544-95db-436b-b82a-106a087ce332 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:35.588463 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552c544-95db-436b-b82a-106a087ce332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:35.590401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552c544-95db-436b-b82a-106a087ce332 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:35.617218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1196/2396] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:07:35 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:07:35.618943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee6184-e69a-4919-8cf5-3e33a98d99ef admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:35.630430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee6184-e69a-4919-8cf5-3e33a98d99ef admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:35.636121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc06dae-2739-4ab7-a30c-162ce7b10f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:35.636959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc06dae-2739-4ab7-a30c-162ce7b10f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:35.637342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc06dae-2739-4ab7-a30c-162ce7b10f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:35.637808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc06dae-2739-4ab7-a30c-162ce7b10f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:35.638503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc06dae-2739-4ab7-a30c-162ce7b10f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:35.640318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc06dae-2739-4ab7-a30c-162ce7b10f5a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:35.651333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1201/2397] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:07:35.656643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552c544-95db-436b-b82a-106a087ce332 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:35.668950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552c544-95db-436b-b82a-106a087ce332 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:35.675936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a231a-233b-4aed-8f68-8a2b701a881b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:35.676613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a231a-233b-4aed-8f68-8a2b701a881b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:35.676894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a231a-233b-4aed-8f68-8a2b701a881b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:35.677165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a231a-233b-4aed-8f68-8a2b701a881b admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:35.677879 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a231a-233b-4aed-8f68-8a2b701a881b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:35.679491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a231a-233b-4aed-8f68-8a2b701a881b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:35.696021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1197/2398] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:07:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:07:35.697027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc06dae-2739-4ab7-a30c-162ce7b10f5a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:35.713425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc06dae-2739-4ab7-a30c-162ce7b10f5a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:35.723348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cd74b-bd92-41e2-a4bb-389d58c91877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:35.724587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd74b-bd92-41e2-a4bb-389d58c91877 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:35.725154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd74b-bd92-41e2-a4bb-389d58c91877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:35.725628 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd74b-bd92-41e2-a4bb-389d58c91877 admin admin] PATH_INFO: `/v3/projects/8d9dc87e8f374f53a28ac79d6e005f77/users/f11a4d79fc334fffb423751e3b410e78/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:35.748195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3cd74b-bd92-41e2-a4bb-389d58c91877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9dc87e8f374f53a28ac79d6e005f77, user_id=f11a4d79fc334fffb423751e3b410e78, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:35.751168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3cd74b-bd92-41e2-a4bb-389d58c91877 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:35.771834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1202/2399] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:07:35 2022] PUT /identity/v3/projects/8d9dc87e8f374f53a28ac79d6e005f77/users/f11a4d79fc334fffb423751e3b410e78/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:07:35.782758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82f56d-396f-40b3-a985-182bac3f646e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:35.784201 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82f56d-396f-40b3-a985-182bac3f646e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:35.784554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82f56d-396f-40b3-a985-182bac3f646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:35.820346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-5b82f56d-396f-40b3-a985-182bac3f646e None None] MFA Rules not processed for user `f11a4d79fc334fffb423751e3b410e78`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:35.821516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5b82f56d-396f-40b3-a985-182bac3f646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:35.913978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1198/2400] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:35.927193 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cd74b-bd92-41e2-a4bb-389d58c91877 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:35.929833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-db3cd74b-bd92-41e2-a4bb-389d58c91877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:35.977146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:36.032029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cd74b-bd92-41e2-a4bb-389d58c91877 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:36.036050 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5eaa3c-c95e-48b7-b644-8ac1a373fa88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:36.036722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eaa3c-c95e-48b7-b644-8ac1a373fa88 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:36.037013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eaa3c-c95e-48b7-b644-8ac1a373fa88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:36.037285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5eaa3c-c95e-48b7-b644-8ac1a373fa88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:36.046833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5eaa3c-c95e-48b7-b644-8ac1a373fa88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:36.048371 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5eaa3c-c95e-48b7-b644-8ac1a373fa88 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:36.058390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1203/2401] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:07:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:07:36.322190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b8da1-ebce-4f00-90db-1adfe6dedc9b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:36.323352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b8da1-ebce-4f00-90db-1adfe6dedc9b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:36.323913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b8da1-ebce-4f00-90db-1adfe6dedc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:36.354851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ab9b8da1-ebce-4f00-90db-1adfe6dedc9b None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:36.362854 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ab9b8da1-ebce-4f00-90db-1adfe6dedc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:36.396017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:36.481693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1199/2402] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:07:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:07:36.556854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5eaa3c-c95e-48b7-b644-8ac1a373fa88 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:36.558398 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9f5eaa3c-c95e-48b7-b644-8ac1a373fa88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:36.676550 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5eaa3c-c95e-48b7-b644-8ac1a373fa88 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:36.691516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54a2cd-4117-4eeb-be6f-facf1c0c93b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:36.692498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54a2cd-4117-4eeb-be6f-facf1c0c93b6 service swift] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:36.692925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54a2cd-4117-4eeb-be6f-facf1c0c93b6 service swift] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:36.693370 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54a2cd-4117-4eeb-be6f-facf1c0c93b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:36.713010 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54a2cd-4117-4eeb-be6f-facf1c0c93b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:36.714757 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54a2cd-4117-4eeb-be6f-facf1c0c93b6 service swift] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:36.728980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1204/2403] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:07:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:07:46.471197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=17 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:46.723342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=49 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:49.097064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5109c84-1a96-446f-aa9f-da6a4f5705ee None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:49.097648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5109c84-1a96-446f-aa9f-da6a4f5705ee None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:49.097648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5109c84-1a96-446f-aa9f-da6a4f5705ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:49.120517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d5109c84-1a96-446f-aa9f-da6a4f5705ee None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:49.121732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d5109c84-1a96-446f-aa9f-da6a4f5705ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:49.136125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1200/2404] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:07:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:49.154960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54a2cd-4117-4eeb-be6f-facf1c0c93b6 service swift] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:49.166386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54a2cd-4117-4eeb-be6f-facf1c0c93b6 service swift] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:49.174230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79149abb-adf6-424b-84b9-f4b0682e94b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:49.174843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149abb-adf6-424b-84b9-f4b0682e94b9 service swift] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:49.175123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149abb-adf6-424b-84b9-f4b0682e94b9 service swift] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:49.175392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79149abb-adf6-424b-84b9-f4b0682e94b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:49.184773 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79149abb-adf6-424b-84b9-f4b0682e94b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:49.186391 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79149abb-adf6-424b-84b9-f4b0682e94b9 service swift] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:49.196144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1205/2405] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:07:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:07:51.658083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5109c84-1a96-446f-aa9f-da6a4f5705ee None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:51.880372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5109c84-1a96-446f-aa9f-da6a4f5705ee None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:51.886672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:51.887747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:51.888542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:51.889096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] PATH_INFO: `/v3/users/f11a4d79fc334fffb423751e3b410e78` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:51.891686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f11a4d79fc334fffb423751e3b410e78)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:51.897539 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:52.036494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f11a4d79fc334fffb423751e3b410e78'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:52.053943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f11a4d79fc334fffb423751e3b410e78'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:52.063854 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f11a4d79fc334fffb423751e3b410e78'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:52.064401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f11a4d79fc334fffb423751e3b410e78'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:52.065030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f11a4d79fc334fffb423751e3b410e78'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:52.087865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1201/2406] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:07:51 2022] DELETE /identity/v3/users/f11a4d79fc334fffb423751e3b410e78 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:07:52.092566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ea68a-96c7-4494-b865-93fb255266bb None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:52.093078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ea68a-96c7-4494-b865-93fb255266bb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:52.093891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ea68a-96c7-4494-b865-93fb255266bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:52.134405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-372ea68a-96c7-4494-b865-93fb255266bb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:07:52.135899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-372ea68a-96c7-4494-b865-93fb255266bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:52.230437 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1206/2407] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:07:52.251925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:52.253470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:52.565809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dbd52-50f0-471e-846b-338ab6beb75d admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:52.577804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fcdb3-28ab-47b1-a0dd-ddbf222493f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:52.580428 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fcdb3-28ab-47b1-a0dd-ddbf222493f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:52.585306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fcdb3-28ab-47b1-a0dd-ddbf222493f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:52.585798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fcdb3-28ab-47b1-a0dd-ddbf222493f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:52.611764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fcdb3-28ab-47b1-a0dd-ddbf222493f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:52.614029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fcdb3-28ab-47b1-a0dd-ddbf222493f6 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:52.663518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1202/2408] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:07:53.116862 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ea68a-96c7-4494-b865-93fb255266bb None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:53.120877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-372ea68a-96c7-4494-b865-93fb255266bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:53.193049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ea68a-96c7-4494-b865-93fb255266bb None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:53.203002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:53.210877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:53.211399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:53.211842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] PATH_INFO: `/v3/projects/8d9dc87e8f374f53a28ac79d6e005f77` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:53.213590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9dc87e8f374f53a28ac79d6e005f77)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:53.216669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:53.286909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fcdb3-28ab-47b1-a0dd-ddbf222493f6 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:07:53.288502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-106fcdb3-28ab-47b1-a0dd-ddbf222493f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:53.333754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:07:53.335662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9dc87e8f374f53a28ac79d6e005f77', 'request_id': 'req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:53.350082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9dc87e8f374f53a28ac79d6e005f77', 'request_id': 'req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:53.369930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9dc87e8f374f53a28ac79d6e005f77', 'request_id': 'req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:53.390512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] The token cache is being invalidate because project 8d9dc87e8f374f53a28ac79d6e005f77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:07:53.391145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:53.392378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-88e8bb77-236b-452a-ac1b-a2d6ea5ff5df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:07:53.398482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1207/2409] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:07:53 2022] DELETE /identity/v3/projects/8d9dc87e8f374f53a28ac79d6e005f77 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:07:53.436106 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fcdb3-28ab-47b1-a0dd-ddbf222493f6 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:07:53.541704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8d0ef6-b926-4c9d-a97d-366232f03514 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:07:53.547343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0ef6-b926-4c9d-a97d-366232f03514 service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:07:53.547629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0ef6-b926-4c9d-a97d-366232f03514 service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:07:53.547896 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8d0ef6-b926-4c9d-a97d-366232f03514 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:07:53.551227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-da8d0ef6-b926-4c9d-a97d-366232f03514 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:07:53.571052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d0ef6-b926-4c9d-a97d-366232f03514 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:07:53.574200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8d0ef6-b926-4c9d-a97d-366232f03514 service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:07:53.605665 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1203/2410] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:07:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:08:00.280967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a28eb-7949-46a3-8085-fd18c0b4a4d7 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:00.281599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a28eb-7949-46a3-8085-fd18c0b4a4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:00.281599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a28eb-7949-46a3-8085-fd18c0b4a4d7 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:00.282890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1208/2411] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:08:00.289513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5934852-ae5b-4afb-900f-c2c571cd385f None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:00.290126 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5934852-ae5b-4afb-900f-c2c571cd385f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:00.290126 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5934852-ae5b-4afb-900f-c2c571cd385f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:00.312592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c5934852-ae5b-4afb-900f-c2c571cd385f None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:00.313725 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c5934852-ae5b-4afb-900f-c2c571cd385f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:00.437511 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1204/2412] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:08:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:08:00.450530 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a28eb-7949-46a3-8085-fd18c0b4a4d7 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:00.452574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-013a28eb-7949-46a3-8085-fd18c0b4a4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:00.525236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a28eb-7949-46a3-8085-fd18c0b4a4d7 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:00.530870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aae012-793e-45e1-b243-72522e5239be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:00.531731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aae012-793e-45e1-b243-72522e5239be service glance] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:00.532031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aae012-793e-45e1-b243-72522e5239be service glance] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:00.532328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aae012-793e-45e1-b243-72522e5239be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:00.546140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aae012-793e-45e1-b243-72522e5239be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:00.548347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aae012-793e-45e1-b243-72522e5239be service glance] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:00.562235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1209/2413] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:00.665164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883da91-5ba3-48e7-b66d-bfebd5dccaf0 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:00.665486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883da91-5ba3-48e7-b66d-bfebd5dccaf0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:00.665764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5883da91-5ba3-48e7-b66d-bfebd5dccaf0 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:00.667642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1205/2414] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:08:00.675105 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fed7c8-caa2-4d6f-9ad7-dd03557b9008 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:00.676209 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fed7c8-caa2-4d6f-9ad7-dd03557b9008 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:00.676601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fed7c8-caa2-4d6f-9ad7-dd03557b9008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:00.699605 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c9fed7c8-caa2-4d6f-9ad7-dd03557b9008 None None] MFA Rules not processed for user `8dfbda2838b242558c885f913fafb5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:00.700961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c9fed7c8-caa2-4d6f-9ad7-dd03557b9008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:00.711046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:00.767196 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1210/2415] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:08:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:00.782000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883da91-5ba3-48e7-b66d-bfebd5dccaf0 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:00.783702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5883da91-5ba3-48e7-b66d-bfebd5dccaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:00.805946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5883da91-5ba3-48e7-b66d-bfebd5dccaf0 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:00.810941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017c4b7-f581-4638-b635-76deffbe4614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:00.811591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017c4b7-f581-4638-b635-76deffbe4614 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:00.811895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017c4b7-f581-4638-b635-76deffbe4614 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:00.812186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017c4b7-f581-4638-b635-76deffbe4614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:00.822682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017c4b7-f581-4638-b635-76deffbe4614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:00.824350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017c4b7-f581-4638-b635-76deffbe4614 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:00.865503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1206/2416] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:00.994139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8f726-9d3d-4610-b2c9-67647c6f6a43 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:00.995769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8f726-9d3d-4610-b2c9-67647c6f6a43 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:00.996052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8f726-9d3d-4610-b2c9-67647c6f6a43 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:00.998699 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1211/2417] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:08:00 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:08:01.004674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee78cba-dd4e-4749-8260-5711b7b804e4 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:01.005068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee78cba-dd4e-4749-8260-5711b7b804e4 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:01.005420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee78cba-dd4e-4749-8260-5711b7b804e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:01.029678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-eee78cba-dd4e-4749-8260-5711b7b804e4 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:01.032700 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-eee78cba-dd4e-4749-8260-5711b7b804e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:01.053899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1207/2418] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:08:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:08:01.067144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8f726-9d3d-4610-b2c9-67647c6f6a43 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:01.083483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8f726-9d3d-4610-b2c9-67647c6f6a43 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:01.087483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1460a-9585-4f8c-8237-4d63e857b17c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:01.088487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1460a-9585-4f8c-8237-4d63e857b17c service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:01.088723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1460a-9585-4f8c-8237-4d63e857b17c service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:01.088998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1460a-9585-4f8c-8237-4d63e857b17c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:01.100119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1460a-9585-4f8c-8237-4d63e857b17c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:01.101619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1460a-9585-4f8c-8237-4d63e857b17c service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:01.114200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1212/2419] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:02.143732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee78cba-dd4e-4749-8260-5711b7b804e4 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:02.146754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-eee78cba-dd4e-4749-8260-5711b7b804e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:02.162560 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee78cba-dd4e-4749-8260-5711b7b804e4 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:02.169535 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3af5b-4212-4264-982a-fbd6f46e49ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:02.170465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3af5b-4212-4264-982a-fbd6f46e49ad service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:02.170829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3af5b-4212-4264-982a-fbd6f46e49ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:02.171329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3af5b-4212-4264-982a-fbd6f46e49ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:02.173699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-40b3af5b-4212-4264-982a-fbd6f46e49ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:02.197510 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:02.249575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3af5b-4212-4264-982a-fbd6f46e49ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:02.251041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3af5b-4212-4264-982a-fbd6f46e49ad service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:02.262353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1208/2420] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:04.330855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1460a-9585-4f8c-8237-4d63e857b17c service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:04.332496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c6b1460a-9585-4f8c-8237-4d63e857b17c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:04.434485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1460a-9585-4f8c-8237-4d63e857b17c service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:04.439726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:04.440547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:04.440835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:04.441173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] PATH_INFO: `/v3/users/49c4cc6b50a54e5184da195012748c3c` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:04.442515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c4cc6b50a54e5184da195012748c3c)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:04.444654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:04.522528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c4cc6b50a54e5184da195012748c3c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:04.523344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c4cc6b50a54e5184da195012748c3c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:04.548959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c4cc6b50a54e5184da195012748c3c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:04.569876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c4cc6b50a54e5184da195012748c3c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:04.595246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-05097c3b-55d3-44ed-8f96-5c9026047d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c4cc6b50a54e5184da195012748c3c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:04.614772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1213/2421] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:08:04 2022] DELETE /identity/v3/users/49c4cc6b50a54e5184da195012748c3c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:04.615891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64263f2-42c8-433d-a648-996c455c7f40 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:04.616482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64263f2-42c8-433d-a648-996c455c7f40 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:04.616615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64263f2-42c8-433d-a648-996c455c7f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:04.646838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b64263f2-42c8-433d-a648-996c455c7f40 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:04.648784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b64263f2-42c8-433d-a648-996c455c7f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:04.815507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1209/2422] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:04.826238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d751d5e-8af9-45fd-8690-46c65c2931a0 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:04.826781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d751d5e-8af9-45fd-8690-46c65c2931a0 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:04.827163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d751d5e-8af9-45fd-8690-46c65c2931a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:04.842664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64263f2-42c8-433d-a648-996c455c7f40 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:04.844654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b64263f2-42c8-433d-a648-996c455c7f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:04.879723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6d751d5e-8af9-45fd-8690-46c65c2931a0 None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:04.883218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6d751d5e-8af9-45fd-8690-46c65c2931a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:04.934560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:04.958129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64263f2-42c8-433d-a648-996c455c7f40 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:04.970093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e631c76-99c0-4aab-971a-1dfc0fac1e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:04.970893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e631c76-99c0-4aab-971a-1dfc0fac1e5d service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:04.971275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e631c76-99c0-4aab-971a-1dfc0fac1e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:04.971648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e631c76-99c0-4aab-971a-1dfc0fac1e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:04.988117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e631c76-99c0-4aab-971a-1dfc0fac1e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:04.989811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e631c76-99c0-4aab-971a-1dfc0fac1e5d service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:05.017390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1210/2423] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:08:05.018009 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1214/2424] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:05.026165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e631c76-99c0-4aab-971a-1dfc0fac1e5d service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:05.027770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0e631c76-99c0-4aab-971a-1dfc0fac1e5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:05.156900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:05.197443 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e631c76-99c0-4aab-971a-1dfc0fac1e5d service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:05.215640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162a42d-436f-4cbb-8e65-6962350abc59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:05.216339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162a42d-436f-4cbb-8e65-6962350abc59 service swift] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:05.216704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162a42d-436f-4cbb-8e65-6962350abc59 service swift] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:05.216990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162a42d-436f-4cbb-8e65-6962350abc59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:05.240799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162a42d-436f-4cbb-8e65-6962350abc59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:05.243396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162a42d-436f-4cbb-8e65-6962350abc59 service swift] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:05.269516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1211/2425] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:08:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:08:05.481316 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d751d5e-8af9-45fd-8690-46c65c2931a0 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:05.483569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6d751d5e-8af9-45fd-8690-46c65c2931a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:05.545393 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d751d5e-8af9-45fd-8690-46c65c2931a0 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:05.575152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:05.575854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:05.582633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:05.583030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] PATH_INFO: `/v3/projects/e03ce641f1f74868b0363f0edfd30e7d` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:05.585694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03ce641f1f74868b0363f0edfd30e7d)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:05.592394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:05.732515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03ce641f1f74868b0363f0edfd30e7d', 'request_id': 'req-0894e537-0677-4435-8a90-b29e72cb8f03'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:05.747854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03ce641f1f74868b0363f0edfd30e7d', 'request_id': 'req-0894e537-0677-4435-8a90-b29e72cb8f03'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:05.768284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03ce641f1f74868b0363f0edfd30e7d', 'request_id': 'req-0894e537-0677-4435-8a90-b29e72cb8f03'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:05.789701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] The token cache is being invalidate because project e03ce641f1f74868b0363f0edfd30e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:08:05.790001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:05.791661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:05.794902 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1215/2426] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:08:05 2022] DELETE /identity/v3/projects/e03ce641f1f74868b0363f0edfd30e7d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:09.237552 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fca5f2-6de1-47cc-befd-b679bed5bcc7 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:09.238263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fca5f2-6de1-47cc-befd-b679bed5bcc7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:09.238263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fca5f2-6de1-47cc-befd-b679bed5bcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:09.260328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d5fca5f2-6de1-47cc-befd-b679bed5bcc7 None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:09.261614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d5fca5f2-6de1-47cc-befd-b679bed5bcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:09.322142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1212/2427] 10.222.0.49 () {58 vars in 967 bytes} [Mon Aug 1 12:08:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:09.335348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:09.336774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:09.435866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894e537-0677-4435-8a90-b29e72cb8f03 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:09.441090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04076c3-9008-4016-afd1-0f10670f3ad1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:09.441721 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04076c3-9008-4016-afd1-0f10670f3ad1 service swift] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:09.442008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04076c3-9008-4016-afd1-0f10670f3ad1 service swift] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:09.442284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04076c3-9008-4016-afd1-0f10670f3ad1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:09.451469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04076c3-9008-4016-afd1-0f10670f3ad1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:09.453110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04076c3-9008-4016-afd1-0f10670f3ad1 service swift] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:09.461443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1216/2428] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:08:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:08:11.732733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58282e-5cdf-4e1a-a05f-61d2737f0692 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:11.733360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58282e-5cdf-4e1a-a05f-61d2737f0692 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:11.733360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b58282e-5cdf-4e1a-a05f-61d2737f0692 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:11.734658 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1213/2429] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:08:11.763026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6649d-08d3-4554-84c7-703a05d0c598 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:11.764155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6649d-08d3-4554-84c7-703a05d0c598 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:11.764670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6649d-08d3-4554-84c7-703a05d0c598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:11.796512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-bba6649d-08d3-4554-84c7-703a05d0c598 None None] MFA Rules not processed for user `9db663d35f964d42938be0d1a9ed4071`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:11.798230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bba6649d-08d3-4554-84c7-703a05d0c598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:11.842516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58282e-5cdf-4e1a-a05f-61d2737f0692 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:11.844273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6b58282e-5cdf-4e1a-a05f-61d2737f0692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:11.864344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:11.890645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1217/2430] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:08:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:11.907681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6649d-08d3-4554-84c7-703a05d0c598 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:11.909604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bba6649d-08d3-4554-84c7-703a05d0c598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:11.925694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6649d-08d3-4554-84c7-703a05d0c598 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:11.933111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29219fc-ce14-4f2f-b16f-8dabfda0734f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:11.933984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29219fc-ce14-4f2f-b16f-8dabfda0734f service placement] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:11.934352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29219fc-ce14-4f2f-b16f-8dabfda0734f service placement] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:11.934721 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29219fc-ce14-4f2f-b16f-8dabfda0734f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:11.944709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29219fc-ce14-4f2f-b16f-8dabfda0734f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:11.946503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29219fc-ce14-4f2f-b16f-8dabfda0734f service placement] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:11.955857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b58282e-5cdf-4e1a-a05f-61d2737f0692 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:11.958579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1218/2431] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:11.960450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:11.961451 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:11.961744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:11.962043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] PATH_INFO: `/v3/users/f82ce1e8ee7d441e9365de3caf8f6c6f` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:11.963245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f82ce1e8ee7d441e9365de3caf8f6c6f)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:11.965248 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:12.012495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f82ce1e8ee7d441e9365de3caf8f6c6f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:12.024708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f82ce1e8ee7d441e9365de3caf8f6c6f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:12.031838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f82ce1e8ee7d441e9365de3caf8f6c6f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:12.032603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f82ce1e8ee7d441e9365de3caf8f6c6f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:12.033304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-2fb21f9f-05a3-4553-9e32-bb53c5aaffa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f82ce1e8ee7d441e9365de3caf8f6c6f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:12.048026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1214/2432] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:08:11 2022] DELETE /identity/v3/users/f82ce1e8ee7d441e9365de3caf8f6c6f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:08:12.293003 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29219fc-ce14-4f2f-b16f-8dabfda0734f service placement] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:12.308820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29219fc-ce14-4f2f-b16f-8dabfda0734f service placement] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:12.372584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:12.373376 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:12.373719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:12.374062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] PATH_INFO: `/v3/projects/b0422eb0a2a74ae099b22570c84afbbb` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:12.375329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0422eb0a2a74ae099b22570c84afbbb)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:12.378099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:12.427987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0422eb0a2a74ae099b22570c84afbbb', 'request_id': 'req-b9ec37e7-4155-4bd7-9315-1f456ba2e334'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:12.434157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0422eb0a2a74ae099b22570c84afbbb', 'request_id': 'req-b9ec37e7-4155-4bd7-9315-1f456ba2e334'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:12.442188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0422eb0a2a74ae099b22570c84afbbb', 'request_id': 'req-b9ec37e7-4155-4bd7-9315-1f456ba2e334'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:12.453707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] The token cache is being invalidate because project b0422eb0a2a74ae099b22570c84afbbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:08:12.453995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:12.454645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ec37e7-4155-4bd7-9315-1f456ba2e334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:12.456213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1219/2433] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:08:12 2022] DELETE /identity/v3/projects/b0422eb0a2a74ae099b22570c84afbbb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:12.462298 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5e5d2-0845-4e5a-8c08-06444e2ea1a4 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:12.462844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5e5d2-0845-4e5a-8c08-06444e2ea1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:12.463125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5e5d2-0845-4e5a-8c08-06444e2ea1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:12.483940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-0ab5e5d2-0845-4e5a-8c08-06444e2ea1a4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:12.485211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0ab5e5d2-0845-4e5a-8c08-06444e2ea1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:12.506358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:12.579126 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1215/2434] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:12.587121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259c27f-3ee2-4ba4-9bbe-d3848922f984 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:12.587402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259c27f-3ee2-4ba4-9bbe-d3848922f984 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:12.587677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259c27f-3ee2-4ba4-9bbe-d3848922f984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:12.612641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7259c27f-3ee2-4ba4-9bbe-d3848922f984 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:12.613861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7259c27f-3ee2-4ba4-9bbe-d3848922f984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:12.623127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1220/2435] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:12.627624 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5e5d2-0845-4e5a-8c08-06444e2ea1a4 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:12.638344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5e5d2-0845-4e5a-8c08-06444e2ea1a4 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:12.644321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf18a78-d832-4422-9997-dcd3ea4c08cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:12.645031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf18a78-d832-4422-9997-dcd3ea4c08cf admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:12.645381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf18a78-d832-4422-9997-dcd3ea4c08cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:12.645737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf18a78-d832-4422-9997-dcd3ea4c08cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:12.646537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf18a78-d832-4422-9997-dcd3ea4c08cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:08:12.646904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf18a78-d832-4422-9997-dcd3ea4c08cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:12.648747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf18a78-d832-4422-9997-dcd3ea4c08cf admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:12.661410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1216/2436] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:08:12.666040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259c27f-3ee2-4ba4-9bbe-d3848922f984 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:12.675492 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259c27f-3ee2-4ba4-9bbe-d3848922f984 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:12.680608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db029d3b-90f6-4007-983e-1aa5c04c9b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:12.681155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db029d3b-90f6-4007-983e-1aa5c04c9b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:12.681434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db029d3b-90f6-4007-983e-1aa5c04c9b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:12.681706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db029d3b-90f6-4007-983e-1aa5c04c9b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:12.682361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db029d3b-90f6-4007-983e-1aa5c04c9b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:12.684310 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db029d3b-90f6-4007-983e-1aa5c04c9b67 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:12.719155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1221/2437] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:08:12 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:08:12.720268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf18a78-d832-4422-9997-dcd3ea4c08cf admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:12.731478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf18a78-d832-4422-9997-dcd3ea4c08cf admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:12.790114 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087dacc-6c8b-430b-b3d0-151c3a0d31b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:12.790881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087dacc-6c8b-430b-b3d0-151c3a0d31b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:12.790987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087dacc-6c8b-430b-b3d0-151c3a0d31b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:12.791263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087dacc-6c8b-430b-b3d0-151c3a0d31b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:12.791905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087dacc-6c8b-430b-b3d0-151c3a0d31b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:12.793865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087dacc-6c8b-430b-b3d0-151c3a0d31b8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:12.816746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1217/2438] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:08:12 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:08:12.821109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db029d3b-90f6-4007-983e-1aa5c04c9b67 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:12.832194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db029d3b-90f6-4007-983e-1aa5c04c9b67 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:12.837617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f93d9c-3923-49ac-b1d1-d39830963684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:12.838340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f93d9c-3923-49ac-b1d1-d39830963684 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:12.838534 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f93d9c-3923-49ac-b1d1-d39830963684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:12.838809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f93d9c-3923-49ac-b1d1-d39830963684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:12.839519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f93d9c-3923-49ac-b1d1-d39830963684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:12.841168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f93d9c-3923-49ac-b1d1-d39830963684 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:12.856184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1222/2439] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:08:12.868271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087dacc-6c8b-430b-b3d0-151c3a0d31b8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:12.880895 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087dacc-6c8b-430b-b3d0-151c3a0d31b8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:12.886112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4a2dc-fce6-4a2f-8204-ef56413843c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:12.886885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4a2dc-fce6-4a2f-8204-ef56413843c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:12.887275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4a2dc-fce6-4a2f-8204-ef56413843c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:12.887627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4a2dc-fce6-4a2f-8204-ef56413843c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:12.888469 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4a2dc-fce6-4a2f-8204-ef56413843c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:12.890448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4a2dc-fce6-4a2f-8204-ef56413843c1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:12.908888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1218/2440] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:08:12.910872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f93d9c-3923-49ac-b1d1-d39830963684 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:12.933904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f93d9c-3923-49ac-b1d1-d39830963684 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:12.938612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55610f30-7890-4391-b0f0-34623ac90a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:12.939292 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610f30-7890-4391-b0f0-34623ac90a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:12.939568 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610f30-7890-4391-b0f0-34623ac90a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:12.939838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55610f30-7890-4391-b0f0-34623ac90a00 admin admin] PATH_INFO: `/v3/projects/cbf1e3306e5e4972aea18cbc1b1431e2/users/f8358c97e4f242b5a59ef75c4058d5bc/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:12.955607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55610f30-7890-4391-b0f0-34623ac90a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf1e3306e5e4972aea18cbc1b1431e2, user_id=f8358c97e4f242b5a59ef75c4058d5bc, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:12.958399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55610f30-7890-4391-b0f0-34623ac90a00 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:12.961442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:12.976866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1223/2441] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:08:12 2022] PUT /identity/v3/projects/cbf1e3306e5e4972aea18cbc1b1431e2/users/f8358c97e4f242b5a59ef75c4058d5bc/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:12.987407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b88bf-9b3d-4b73-bafd-8e349ee6b54f None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:12.987702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b88bf-9b3d-4b73-bafd-8e349ee6b54f None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:12.988026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b88bf-9b3d-4b73-bafd-8e349ee6b54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:13.030199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e13b88bf-9b3d-4b73-bafd-8e349ee6b54f None None] MFA Rules not processed for user `f8358c97e4f242b5a59ef75c4058d5bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:13.032209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e13b88bf-9b3d-4b73-bafd-8e349ee6b54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:13.107365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1219/2442] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:13.124630 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55610f30-7890-4391-b0f0-34623ac90a00 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:13.126625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-55610f30-7890-4391-b0f0-34623ac90a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:13.193768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55610f30-7890-4391-b0f0-34623ac90a00 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:13.197348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b1b70-502b-4f34-bb13-a18e1d8bcab2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:13.197977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b1b70-502b-4f34-bb13-a18e1d8bcab2 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:13.198253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b1b70-502b-4f34-bb13-a18e1d8bcab2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:13.198526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b1b70-502b-4f34-bb13-a18e1d8bcab2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:13.207087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b1b70-502b-4f34-bb13-a18e1d8bcab2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:13.208659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b1b70-502b-4f34-bb13-a18e1d8bcab2 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:13.217500 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1224/2443] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:13.246536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b88bf-9b3d-4b73-bafd-8e349ee6b54f None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:13.256633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:13.323862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b88bf-9b3d-4b73-bafd-8e349ee6b54f None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:13.328653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:13.329368 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:13.329740 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:13.330014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] PATH_INFO: `/v3/users/f8358c97e4f242b5a59ef75c4058d5bc` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:13.331206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8358c97e4f242b5a59ef75c4058d5bc)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:13.333207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:13.410781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8358c97e4f242b5a59ef75c4058d5bc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:13.425570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8358c97e4f242b5a59ef75c4058d5bc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:13.432682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8358c97e4f242b5a59ef75c4058d5bc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:13.433447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8358c97e4f242b5a59ef75c4058d5bc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:13.434050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8358c97e4f242b5a59ef75c4058d5bc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:13.448003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1220/2444] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:08:13 2022] DELETE /identity/v3/users/f8358c97e4f242b5a59ef75c4058d5bc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:13.452694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5e69c-2e46-451b-8f44-3a64fbb0a360 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:13.452976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5e69c-2e46-451b-8f44-3a64fbb0a360 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:13.453255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f5e69c-2e46-451b-8f44-3a64fbb0a360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:13.472980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-17f5e69c-2e46-451b-8f44-3a64fbb0a360 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:13.474460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-17f5e69c-2e46-451b-8f44-3a64fbb0a360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:13.539143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1225/2445] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:08:13.550145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:13.552337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:13.640066 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503052a-b3bd-49d1-b3eb-b90505319a34 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:13.643827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19935745-63e3-4bae-aec0-dd184bee485d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:13.644533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19935745-63e3-4bae-aec0-dd184bee485d service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:13.644815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19935745-63e3-4bae-aec0-dd184bee485d service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:13.645090 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19935745-63e3-4bae-aec0-dd184bee485d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:13.654531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19935745-63e3-4bae-aec0-dd184bee485d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:13.656075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19935745-63e3-4bae-aec0-dd184bee485d service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:13.665534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1221/2446] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:13.999907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5e69c-2e46-451b-8f44-3a64fbb0a360 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:14.002275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-17f5e69c-2e46-451b-8f44-3a64fbb0a360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:14.024993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f5e69c-2e46-451b-8f44-3a64fbb0a360 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:14.040417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:14.041389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.041903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.042388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] PATH_INFO: `/v3/projects/cbf1e3306e5e4972aea18cbc1b1431e2` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.044284 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf1e3306e5e4972aea18cbc1b1431e2)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:14.046823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:14.103393 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf1e3306e5e4972aea18cbc1b1431e2', 'request_id': 'req-eaec7bf1-b794-4112-bd53-d58bb70a56fd'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:14.110231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf1e3306e5e4972aea18cbc1b1431e2', 'request_id': 'req-eaec7bf1-b794-4112-bd53-d58bb70a56fd'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:14.119505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf1e3306e5e4972aea18cbc1b1431e2', 'request_id': 'req-eaec7bf1-b794-4112-bd53-d58bb70a56fd'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:14.132892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] The token cache is being invalidate because project cbf1e3306e5e4972aea18cbc1b1431e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:08:14.133179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:14.133840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-eaec7bf1-b794-4112-bd53-d58bb70a56fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:14.135649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1226/2447] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:08:13 2022] DELETE /identity/v3/projects/cbf1e3306e5e4972aea18cbc1b1431e2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:08:14.140587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837558e-af1a-42a4-bfd1-43ab86bb8f5b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.141161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837558e-af1a-42a4-bfd1-43ab86bb8f5b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.141161 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837558e-af1a-42a4-bfd1-43ab86bb8f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.163672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-2837558e-af1a-42a4-bfd1-43ab86bb8f5b None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:14.164969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2837558e-af1a-42a4-bfd1-43ab86bb8f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:14.195984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:14.229754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1222/2448] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:08:14.239582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987f82a-6b01-4ff6-8b61-6ccd5b8b83f6 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.239863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987f82a-6b01-4ff6-8b61-6ccd5b8b83f6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.240141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987f82a-6b01-4ff6-8b61-6ccd5b8b83f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.280752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-0987f82a-6b01-4ff6-8b61-6ccd5b8b83f6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:14.282133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0987f82a-6b01-4ff6-8b61-6ccd5b8b83f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:14.293374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1227/2449] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:14.299422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837558e-af1a-42a4-bfd1-43ab86bb8f5b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:14.309902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2837558e-af1a-42a4-bfd1-43ab86bb8f5b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:14.314696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d857763-fb40-43ac-ba82-031b0f01fcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:14.315292 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d857763-fb40-43ac-ba82-031b0f01fcc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.315571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d857763-fb40-43ac-ba82-031b0f01fcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.315846 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d857763-fb40-43ac-ba82-031b0f01fcc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.316559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d857763-fb40-43ac-ba82-031b0f01fcc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:08:14.316888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d857763-fb40-43ac-ba82-031b0f01fcc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:14.318513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d857763-fb40-43ac-ba82-031b0f01fcc4 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:14.330925 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1223/2450] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:08:14.335398 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987f82a-6b01-4ff6-8b61-6ccd5b8b83f6 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:14.346021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987f82a-6b01-4ff6-8b61-6ccd5b8b83f6 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:14.355533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ec606-b024-4ac4-a382-803fe44b5257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:14.356963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ec606-b024-4ac4-a382-803fe44b5257 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.357515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ec606-b024-4ac4-a382-803fe44b5257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.358077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ec606-b024-4ac4-a382-803fe44b5257 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.359060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ec606-b024-4ac4-a382-803fe44b5257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:14.361587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ec606-b024-4ac4-a382-803fe44b5257 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:14.382842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:14.403386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1228/2451] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:08:14 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:08:14.407817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d857763-fb40-43ac-ba82-031b0f01fcc4 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:14.418079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d857763-fb40-43ac-ba82-031b0f01fcc4 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:14.510452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29560a3-1ecf-42e1-b662-66ce37b388a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:14.511409 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29560a3-1ecf-42e1-b662-66ce37b388a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.511853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29560a3-1ecf-42e1-b662-66ce37b388a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.512470 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29560a3-1ecf-42e1-b662-66ce37b388a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.512838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29560a3-1ecf-42e1-b662-66ce37b388a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:14.514885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29560a3-1ecf-42e1-b662-66ce37b388a9 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:14.539242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1224/2452] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:08:14 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:08:14.543652 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ec606-b024-4ac4-a382-803fe44b5257 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:14.556677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ec606-b024-4ac4-a382-803fe44b5257 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:14.561758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38466f-7ea9-427f-8c9e-be881ba97b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:14.562784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38466f-7ea9-427f-8c9e-be881ba97b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.562784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38466f-7ea9-427f-8c9e-be881ba97b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.562964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38466f-7ea9-427f-8c9e-be881ba97b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.564694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38466f-7ea9-427f-8c9e-be881ba97b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:14.566432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38466f-7ea9-427f-8c9e-be881ba97b97 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:14.591863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1229/2453] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:08:14.593448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29560a3-1ecf-42e1-b662-66ce37b388a9 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:14.614683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29560a3-1ecf-42e1-b662-66ce37b388a9 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:14.620882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef8774-282a-4ef2-8dd6-c57e8c6cb000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:14.621986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef8774-282a-4ef2-8dd6-c57e8c6cb000 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.622432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef8774-282a-4ef2-8dd6-c57e8c6cb000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.622940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef8774-282a-4ef2-8dd6-c57e8c6cb000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.624494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef8774-282a-4ef2-8dd6-c57e8c6cb000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:14.626421 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ef8774-282a-4ef2-8dd6-c57e8c6cb000 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:14.644328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1225/2454] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:08:14.646541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38466f-7ea9-427f-8c9e-be881ba97b97 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:14.658357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38466f-7ea9-427f-8c9e-be881ba97b97 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:14.663501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea1dd7-f6ed-4da4-a3fd-2ec49189625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:14.664201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea1dd7-f6ed-4da4-a3fd-2ec49189625e admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.664514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea1dd7-f6ed-4da4-a3fd-2ec49189625e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.664797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea1dd7-f6ed-4da4-a3fd-2ec49189625e admin admin] PATH_INFO: `/v3/projects/e95dc4d3587d44bc96fa11ec595d1947/users/1bf653a66a6e466eb28814f155ca6fb2/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.682484 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea1dd7-f6ed-4da4-a3fd-2ec49189625e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95dc4d3587d44bc96fa11ec595d1947, user_id=1bf653a66a6e466eb28814f155ca6fb2, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:14.685261 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea1dd7-f6ed-4da4-a3fd-2ec49189625e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:14.700810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1230/2455] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:08:14 2022] PUT /identity/v3/projects/e95dc4d3587d44bc96fa11ec595d1947/users/1bf653a66a6e466eb28814f155ca6fb2/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:14.712409 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62510f07-4e33-43c6-bfa4-4576039215b7 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.712966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62510f07-4e33-43c6-bfa4-4576039215b7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.713342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62510f07-4e33-43c6-bfa4-4576039215b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.742838 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-62510f07-4e33-43c6-bfa4-4576039215b7 None None] MFA Rules not processed for user `1bf653a66a6e466eb28814f155ca6fb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:14.744096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-62510f07-4e33-43c6-bfa4-4576039215b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:14.822579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1226/2456] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:14.837160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea1dd7-f6ed-4da4-a3fd-2ec49189625e admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:14.844224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-baea1dd7-f6ed-4da4-a3fd-2ec49189625e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:14.927737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea1dd7-f6ed-4da4-a3fd-2ec49189625e admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:14.932127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b8001-8551-4deb-abde-b5bcea76d670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:14.932789 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b8001-8551-4deb-abde-b5bcea76d670 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:14.933208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b8001-8551-4deb-abde-b5bcea76d670 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:14.933469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b8001-8551-4deb-abde-b5bcea76d670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:14.943480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b8001-8551-4deb-abde-b5bcea76d670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:14.944976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b8001-8551-4deb-abde-b5bcea76d670 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:14.966399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1231/2457] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:24.817288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:24.951028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:25.219279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62510f07-4e33-43c6-bfa4-4576039215b7 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:25.290163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62510f07-4e33-43c6-bfa4-4576039215b7 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:25.295588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:25.296402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:25.296699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:25.296981 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] PATH_INFO: `/v3/users/1bf653a66a6e466eb28814f155ca6fb2` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:25.299336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf653a66a6e466eb28814f155ca6fb2)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:25.301433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:25.363720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf653a66a6e466eb28814f155ca6fb2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:25.380388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf653a66a6e466eb28814f155ca6fb2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:25.403399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf653a66a6e466eb28814f155ca6fb2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:25.403399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf653a66a6e466eb28814f155ca6fb2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:25.403399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf653a66a6e466eb28814f155ca6fb2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:25.407316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1227/2458] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:08:25 2022] DELETE /identity/v3/users/1bf653a66a6e466eb28814f155ca6fb2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:25.411916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37e0a9-8be9-4410-87db-1f4b84843186 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:25.412307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37e0a9-8be9-4410-87db-1f4b84843186 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:25.412693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a37e0a9-8be9-4410-87db-1f4b84843186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:25.431707 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6a37e0a9-8be9-4410-87db-1f4b84843186 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:25.433057 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6a37e0a9-8be9-4410-87db-1f4b84843186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:25.495681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1232/2459] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:25.505274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:25.507666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:25.566460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b17156d-5607-42b6-961b-df378e9e432d admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:25.570151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea6557-c2fe-460f-9d62-90e592a48a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:25.571092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea6557-c2fe-460f-9d62-90e592a48a44 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:25.571092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea6557-c2fe-460f-9d62-90e592a48a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:25.571324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea6557-c2fe-460f-9d62-90e592a48a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:25.581282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea6557-c2fe-460f-9d62-90e592a48a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:25.582862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ea6557-c2fe-460f-9d62-90e592a48a44 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:25.593541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1228/2460] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:25.870804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37e0a9-8be9-4410-87db-1f4b84843186 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:25.872456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6a37e0a9-8be9-4410-87db-1f4b84843186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:25.884992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a37e0a9-8be9-4410-87db-1f4b84843186 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:25.891703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:25.892455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:25.892775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:25.893092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] PATH_INFO: `/v3/projects/e95dc4d3587d44bc96fa11ec595d1947` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:25.894650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95dc4d3587d44bc96fa11ec595d1947)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:25.896887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:25.947157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95dc4d3587d44bc96fa11ec595d1947', 'request_id': 'req-1581e23f-7b3a-4e12-81bd-2224a37b614f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:25.953453 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95dc4d3587d44bc96fa11ec595d1947', 'request_id': 'req-1581e23f-7b3a-4e12-81bd-2224a37b614f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:25.961786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95dc4d3587d44bc96fa11ec595d1947', 'request_id': 'req-1581e23f-7b3a-4e12-81bd-2224a37b614f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:25.972519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] The token cache is being invalidate because project e95dc4d3587d44bc96fa11ec595d1947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:08:25.972822 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:25.973518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1581e23f-7b3a-4e12-81bd-2224a37b614f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:25.975142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1233/2461] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:08:25 2022] DELETE /identity/v3/projects/e95dc4d3587d44bc96fa11ec595d1947 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:25.980496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b50a4-899c-4a9d-a39d-7fbbd34296cb None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:25.981046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b50a4-899c-4a9d-a39d-7fbbd34296cb None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:25.981046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b50a4-899c-4a9d-a39d-7fbbd34296cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:26.003613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-787b50a4-899c-4a9d-a39d-7fbbd34296cb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:26.005395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-787b50a4-899c-4a9d-a39d-7fbbd34296cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:26.024717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:26.071930 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1229/2462] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:26.081092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4c862-8be2-4300-98e9-efe0b47788b5 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:26.081396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4c862-8be2-4300-98e9-efe0b47788b5 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:26.081675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c4c862-8be2-4300-98e9-efe0b47788b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:26.101732 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-30c4c862-8be2-4300-98e9-efe0b47788b5 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:26.103091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-30c4c862-8be2-4300-98e9-efe0b47788b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:26.111254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1234/2463] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:26.116038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b50a4-899c-4a9d-a39d-7fbbd34296cb None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:26.129789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b50a4-899c-4a9d-a39d-7fbbd34296cb None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:26.134999 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6829ce4-dc22-438f-a459-11f4d6abbb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:26.135653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6829ce4-dc22-438f-a459-11f4d6abbb75 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:26.135922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6829ce4-dc22-438f-a459-11f4d6abbb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:26.136243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6829ce4-dc22-438f-a459-11f4d6abbb75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:26.136976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6829ce4-dc22-438f-a459-11f4d6abbb75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:08:26.137279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6829ce4-dc22-438f-a459-11f4d6abbb75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:26.138909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6829ce4-dc22-438f-a459-11f4d6abbb75 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:26.150915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1230/2464] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:08:26.155531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4c862-8be2-4300-98e9-efe0b47788b5 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:26.165650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c4c862-8be2-4300-98e9-efe0b47788b5 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:26.171603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ec99e-5139-4292-bc23-c58dec31837a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:26.172218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ec99e-5139-4292-bc23-c58dec31837a admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:26.172526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ec99e-5139-4292-bc23-c58dec31837a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:26.172793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ec99e-5139-4292-bc23-c58dec31837a admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:26.173456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ec99e-5139-4292-bc23-c58dec31837a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:26.175491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ec99e-5139-4292-bc23-c58dec31837a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:26.206800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1235/2465] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:08:26 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:08:26.211502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6829ce4-dc22-438f-a459-11f4d6abbb75 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:26.221662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6829ce4-dc22-438f-a459-11f4d6abbb75 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:26.282112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c08b5-2879-4d70-97e5-f769021fce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:26.282643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c08b5-2879-4d70-97e5-f769021fce45 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:26.282923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c08b5-2879-4d70-97e5-f769021fce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:26.283193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c08b5-2879-4d70-97e5-f769021fce45 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:26.283856 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c08b5-2879-4d70-97e5-f769021fce45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:26.285924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c08b5-2879-4d70-97e5-f769021fce45 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:26.308553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1231/2466] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:08:26 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:08:26.312930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ec99e-5139-4292-bc23-c58dec31837a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:26.323824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ec99e-5139-4292-bc23-c58dec31837a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:26.328631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b27b1-b020-43fe-83a3-77d487dbfcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:26.329271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b27b1-b020-43fe-83a3-77d487dbfcce admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:26.329548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b27b1-b020-43fe-83a3-77d487dbfcce admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:26.329819 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b27b1-b020-43fe-83a3-77d487dbfcce admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:26.330522 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b27b1-b020-43fe-83a3-77d487dbfcce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:26.332143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b27b1-b020-43fe-83a3-77d487dbfcce admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:26.342468 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1236/2467] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:08:26.346668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c08b5-2879-4d70-97e5-f769021fce45 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:26.360384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c08b5-2879-4d70-97e5-f769021fce45 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:26.369255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef29ec-59d4-4a23-88a5-5631699a800f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:26.369881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef29ec-59d4-4a23-88a5-5631699a800f admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:26.370153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef29ec-59d4-4a23-88a5-5631699a800f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:26.370426 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef29ec-59d4-4a23-88a5-5631699a800f admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:26.371129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef29ec-59d4-4a23-88a5-5631699a800f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:26.372802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef29ec-59d4-4a23-88a5-5631699a800f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:26.384343 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1232/2468] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:08:26.389088 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b27b1-b020-43fe-83a3-77d487dbfcce admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:26.400917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b27b1-b020-43fe-83a3-77d487dbfcce admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:26.406905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1b74-06f7-4cf6-b1a8-656c62234856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:26.407584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1b74-06f7-4cf6-b1a8-656c62234856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:26.407853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1b74-06f7-4cf6-b1a8-656c62234856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:26.408110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be1b74-06f7-4cf6-b1a8-656c62234856 admin admin] PATH_INFO: `/v3/projects/c8270f755c61412ba3f867925da8fde9/users/c6e8c16685904c55a07859af60f11b16/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:26.424836 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be1b74-06f7-4cf6-b1a8-656c62234856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8270f755c61412ba3f867925da8fde9, user_id=c6e8c16685904c55a07859af60f11b16, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:26.427732 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be1b74-06f7-4cf6-b1a8-656c62234856 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:26.443556 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1237/2469] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:08:26 2022] PUT /identity/v3/projects/c8270f755c61412ba3f867925da8fde9/users/c6e8c16685904c55a07859af60f11b16/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:26.455027 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb3806-d813-47ec-b7cf-9ee5f65622bb None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:26.455313 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb3806-d813-47ec-b7cf-9ee5f65622bb None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:26.455583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeb3806-d813-47ec-b7cf-9ee5f65622bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:26.485804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-fbeb3806-d813-47ec-b7cf-9ee5f65622bb None None] MFA Rules not processed for user `c6e8c16685904c55a07859af60f11b16`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:26.487069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fbeb3806-d813-47ec-b7cf-9ee5f65622bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:26.573923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1233/2470] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:26.584665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1b74-06f7-4cf6-b1a8-656c62234856 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:26.586709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a3be1b74-06f7-4cf6-b1a8-656c62234856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:26.609147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:26.647608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be1b74-06f7-4cf6-b1a8-656c62234856 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:26.651475 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a5fdb-73fe-4ea1-8c02-7bcc0a2837ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:26.652232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a5fdb-73fe-4ea1-8c02-7bcc0a2837ac service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:26.652554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a5fdb-73fe-4ea1-8c02-7bcc0a2837ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:26.652835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a5fdb-73fe-4ea1-8c02-7bcc0a2837ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:26.662300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a5fdb-73fe-4ea1-8c02-7bcc0a2837ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:26.663766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9a5fdb-73fe-4ea1-8c02-7bcc0a2837ac service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:26.673295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1238/2471] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:28.057279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca54c27-83f5-4698-8487-dc134664141e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:28.057837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca54c27-83f5-4698-8487-dc134664141e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:28.057837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca54c27-83f5-4698-8487-dc134664141e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:28.077591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-0ca54c27-83f5-4698-8487-dc134664141e None None] MFA Rules not processed for user `c6e8c16685904c55a07859af60f11b16`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:28.078943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0ca54c27-83f5-4698-8487-dc134664141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:28.087600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1234/2472] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:29.700703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab39e8-e41d-4f0e-bbab-0b5f3f222d3d None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:29.700703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab39e8-e41d-4f0e-bbab-0b5f3f222d3d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:29.700703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cab39e8-e41d-4f0e-bbab-0b5f3f222d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:29.731701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2cab39e8-e41d-4f0e-bbab-0b5f3f222d3d None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:29.733040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2cab39e8-e41d-4f0e-bbab-0b5f3f222d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:29.793738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1239/2473] 10.222.0.49 () {58 vars in 967 bytes} [Mon Aug 1 12:08:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:29.808626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca54c27-83f5-4698-8487-dc134664141e None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:29.810227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0ca54c27-83f5-4698-8487-dc134664141e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:29.824588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:29.885726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca54c27-83f5-4698-8487-dc134664141e None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:29.891243 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e1861-bcfd-44af-9f91-b87fa27d49b0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:29.891876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e1861-bcfd-44af-9f91-b87fa27d49b0 service swift] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:29.892156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e1861-bcfd-44af-9f91-b87fa27d49b0 service swift] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:29.892459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e1861-bcfd-44af-9f91-b87fa27d49b0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:29.901532 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e1861-bcfd-44af-9f91-b87fa27d49b0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:29.903119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e1861-bcfd-44af-9f91-b87fa27d49b0 service swift] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:29.911251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1235/2474] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:08:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:08:39.784454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=42 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:39.908307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=27 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:44.303150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2028d56-3228-4983-b1d7-cff4a126ac4d None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:44.303640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2028d56-3228-4983-b1d7-cff4a126ac4d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:44.303793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2028d56-3228-4983-b1d7-cff4a126ac4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:44.325728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-e2028d56-3228-4983-b1d7-cff4a126ac4d None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:44.326958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e2028d56-3228-4983-b1d7-cff4a126ac4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:44.334320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1240/2475] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:08:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:08:44.359460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e1861-bcfd-44af-9f91-b87fa27d49b0 service swift] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:44.372175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e1861-bcfd-44af-9f91-b87fa27d49b0 service swift] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:44.378807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ba619-157b-41b8-a8d9-285652419226 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:44.379428 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ba619-157b-41b8-a8d9-285652419226 service swift] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:44.379755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ba619-157b-41b8-a8d9-285652419226 service swift] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:44.380054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ba619-157b-41b8-a8d9-285652419226 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:44.390714 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ba619-157b-41b8-a8d9-285652419226 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:44.392338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ba619-157b-41b8-a8d9-285652419226 service swift] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:44.401303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1236/2476] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:08:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:08:44.986530 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a63b9-b6db-420c-abf6-a35287a866b4 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:44.986808 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a63b9-b6db-420c-abf6-a35287a866b4 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:44.987078 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a63b9-b6db-420c-abf6-a35287a866b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:45.008646 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-b94a63b9-b6db-420c-abf6-a35287a866b4 None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:45.010411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b94a63b9-b6db-420c-abf6-a35287a866b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:45.022809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1241/2477] 10.222.0.49 () {58 vars in 967 bytes} [Mon Aug 1 12:08:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:45.038866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ba619-157b-41b8-a8d9-285652419226 service swift] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:45.057685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ba619-157b-41b8-a8d9-285652419226 service swift] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:45.064776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c518ab-3fbd-4b90-a184-4dfab2bcd7fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:45.065674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c518ab-3fbd-4b90-a184-4dfab2bcd7fa service swift] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:45.066182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c518ab-3fbd-4b90-a184-4dfab2bcd7fa service swift] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:45.066644 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c518ab-3fbd-4b90-a184-4dfab2bcd7fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:45.076179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c518ab-3fbd-4b90-a184-4dfab2bcd7fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:45.078116 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c518ab-3fbd-4b90-a184-4dfab2bcd7fa service swift] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:45.088311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1237/2478] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:08:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:08:45.377790 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4fc45-fd09-4195-bfcf-0a32c2900a89 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:45.378280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4fc45-fd09-4195-bfcf-0a32c2900a89 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:45.378371 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4fc45-fd09-4195-bfcf-0a32c2900a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:45.399346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-faa4fc45-fd09-4195-bfcf-0a32c2900a89 None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:45.400607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-faa4fc45-fd09-4195-bfcf-0a32c2900a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:45.408881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1242/2479] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:08:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:45.421008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c518ab-3fbd-4b90-a184-4dfab2bcd7fa service swift] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:45.432081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c518ab-3fbd-4b90-a184-4dfab2bcd7fa service swift] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:45.439275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c4273-2937-488d-ad4a-3bd1bc579aca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:45.440165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c4273-2937-488d-ad4a-3bd1bc579aca service swift] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:45.440478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c4273-2937-488d-ad4a-3bd1bc579aca service swift] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:45.440754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c4273-2937-488d-ad4a-3bd1bc579aca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:45.450743 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c4273-2937-488d-ad4a-3bd1bc579aca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:45.452372 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c4273-2937-488d-ad4a-3bd1bc579aca service swift] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:45.460797 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1238/2480] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:08:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:08:45.813390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b200b-fba2-4997-b448-1e7efea0eaa9 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:45.813677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b200b-fba2-4997-b448-1e7efea0eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:45.813997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b200b-fba2-4997-b448-1e7efea0eaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:45.834800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-686b200b-fba2-4997-b448-1e7efea0eaa9 None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:45.836025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-686b200b-fba2-4997-b448-1e7efea0eaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:45.846266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1243/2481] 10.222.0.49 () {58 vars in 967 bytes} [Mon Aug 1 12:08:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:45.864947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c4273-2937-488d-ad4a-3bd1bc579aca service swift] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:45.876980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c4273-2937-488d-ad4a-3bd1bc579aca service swift] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:45.882713 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9ce7d-cb38-4f66-aba7-018e4efe1f6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:45.883332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9ce7d-cb38-4f66-aba7-018e4efe1f6f service swift] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:45.883612 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9ce7d-cb38-4f66-aba7-018e4efe1f6f service swift] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:45.883886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9ce7d-cb38-4f66-aba7-018e4efe1f6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:45.893256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9ce7d-cb38-4f66-aba7-018e4efe1f6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:45.894854 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9ce7d-cb38-4f66-aba7-018e4efe1f6f service swift] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:45.903949 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1239/2482] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:08:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 12:08:48.067073 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b200b-fba2-4997-b448-1e7efea0eaa9 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:48.135550 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b200b-fba2-4997-b448-1e7efea0eaa9 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:48.144885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:48.145788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:48.146183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:48.146569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] PATH_INFO: `/v3/users/c6e8c16685904c55a07859af60f11b16` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:48.148676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e8c16685904c55a07859af60f11b16)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:48.151129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:48.218365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e8c16685904c55a07859af60f11b16'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:48.219133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e8c16685904c55a07859af60f11b16'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:48.227215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e8c16685904c55a07859af60f11b16'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:48.240470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e8c16685904c55a07859af60f11b16'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:48.247846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e8c16685904c55a07859af60f11b16'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:48.253944 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1244/2483] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:08:48 2022] DELETE /identity/v3/users/c6e8c16685904c55a07859af60f11b16 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:48.258566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf95971-0f05-4800-b07b-a9fe6578a9fe None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:48.259053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf95971-0f05-4800-b07b-a9fe6578a9fe None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:48.259155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf95971-0f05-4800-b07b-a9fe6578a9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:48.277952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ebf95971-0f05-4800-b07b-a9fe6578a9fe None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:08:48.279153 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ebf95971-0f05-4800-b07b-a9fe6578a9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:48.354847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1240/2484] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:08:48.364843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:48.366773 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:48.427610 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460dd7-aa92-47bd-bb7e-5c560e95e331 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:48.431582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b087eab-5573-4d7d-a58d-e0113226f521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:48.432239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b087eab-5573-4d7d-a58d-e0113226f521 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:48.432545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b087eab-5573-4d7d-a58d-e0113226f521 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:48.432860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b087eab-5573-4d7d-a58d-e0113226f521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:48.441399 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:48.443896 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b087eab-5573-4d7d-a58d-e0113226f521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:48.445441 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b087eab-5573-4d7d-a58d-e0113226f521 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:48.460534 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1245/2485] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:08:48.649975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf95971-0f05-4800-b07b-a9fe6578a9fe None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:08:48.651469 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ebf95971-0f05-4800-b07b-a9fe6578a9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:08:48.667091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf95971-0f05-4800-b07b-a9fe6578a9fe None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:08:48.672106 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:08:48.673227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:08:48.673227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:08:48.673573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] PATH_INFO: `/v3/projects/c8270f755c61412ba3f867925da8fde9` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:08:48.674577 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8270f755c61412ba3f867925da8fde9)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:08:48.676640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:08:48.725571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8270f755c61412ba3f867925da8fde9', 'request_id': 'req-899bda9e-7610-4d18-896d-05352389b922'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:48.731951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8270f755c61412ba3f867925da8fde9', 'request_id': 'req-899bda9e-7610-4d18-896d-05352389b922'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:48.742071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8270f755c61412ba3f867925da8fde9', 'request_id': 'req-899bda9e-7610-4d18-896d-05352389b922'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:48.752844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] The token cache is being invalidate because project c8270f755c61412ba3f867925da8fde9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:08:48.753129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:48.753749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-899bda9e-7610-4d18-896d-05352389b922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:08:48.755267 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1241/2486] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:08:48 2022] DELETE /identity/v3/projects/c8270f755c61412ba3f867925da8fde9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:08:58.461364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:08:58.749646 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:09:04.175318 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51d6cb-dcee-49f5-bac5-5a4d049b6ba4 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:09:04.175903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51d6cb-dcee-49f5-bac5-5a4d049b6ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:09:04.175903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51d6cb-dcee-49f5-bac5-5a4d049b6ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:09:04.196227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-fe51d6cb-dcee-49f5-bac5-5a4d049b6ba4 None None] MFA Rules not processed for user `fc8a5fa04b6141329778221169afdfc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:09:04.197479 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-fe51d6cb-dcee-49f5-bac5-5a4d049b6ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:09:04.263752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1246/2487] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:09:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:09:14.254176 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=25 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:09:20.694581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ef996-2c54-4797-ade3-fc53f25bef4b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:09:20.695174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ef996-2c54-4797-ade3-fc53f25bef4b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:09:20.695174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ef996-2c54-4797-ade3-fc53f25bef4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:09:20.716413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a89ef996-2c54-4797-ade3-fc53f25bef4b None None] MFA Rules not processed for user `e98a16b117294e10b9d0c8c721d8e114`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:09:20.718052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a89ef996-2c54-4797-ade3-fc53f25bef4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:09:20.806833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1242/2488] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:09:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:09:30.795293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=25 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:09:43.447390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e49fb-b977-4e0c-834b-f4d67dd726e6 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:09:43.448414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e49fb-b977-4e0c-834b-f4d67dd726e6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:09:43.448414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e49fb-b977-4e0c-834b-f4d67dd726e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:09:43.468763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-773e49fb-b977-4e0c-834b-f4d67dd726e6 None None] MFA Rules not processed for user `443696f19aec46fc9cd59bd9d61587f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:09:43.470008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-773e49fb-b977-4e0c-834b-f4d67dd726e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:09:43.532915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1247/2489] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:09:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:09:51.437873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ef996-2c54-4797-ade3-fc53f25bef4b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:09:51.440155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a89ef996-2c54-4797-ade3-fc53f25bef4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:09:51.499631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ef996-2c54-4797-ade3-fc53f25bef4b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:09:51.503602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ab37f-f6a2-4b2f-a218-b1193ae00680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:09:51.504230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab37f-f6a2-4b2f-a218-b1193ae00680 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:09:51.504541 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab37f-f6a2-4b2f-a218-b1193ae00680 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:09:51.504817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ab37f-f6a2-4b2f-a218-b1193ae00680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:09:51.506823 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3d7ab37f-f6a2-4b2f-a218-b1193ae00680 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:09:51.518804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ab37f-f6a2-4b2f-a218-b1193ae00680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:09:51.520374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ab37f-f6a2-4b2f-a218-b1193ae00680 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:09:51.539327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1243/2490] 10.222.0.49 () {60 vars in 1396 bytes} [Mon Aug 1 12:09:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:09:53.525058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=25 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:01.534026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=26 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:02.645480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e49fb-b977-4e0c-834b-f4d67dd726e6 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:02.655675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e49fb-b977-4e0c-834b-f4d67dd726e6 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:02.660273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014374d7-c5bf-48c9-9ceb-d538cfc2a60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:02.660913 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014374d7-c5bf-48c9-9ceb-d538cfc2a60e service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:02.661194 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014374d7-c5bf-48c9-9ceb-d538cfc2a60e service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:02.661465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014374d7-c5bf-48c9-9ceb-d538cfc2a60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:02.663458 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-014374d7-c5bf-48c9-9ceb-d538cfc2a60e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:02.749083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014374d7-c5bf-48c9-9ceb-d538cfc2a60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:02.750431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014374d7-c5bf-48c9-9ceb-d538cfc2a60e service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:02.766906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1248/2491] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:03.955078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ab37f-f6a2-4b2f-a218-b1193ae00680 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:03.964230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ab37f-f6a2-4b2f-a218-b1193ae00680 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:03.968447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe95dc9-0b1b-4fb6-9c75-2db4905e8621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:03.969060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe95dc9-0b1b-4fb6-9c75-2db4905e8621 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:03.969338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe95dc9-0b1b-4fb6-9c75-2db4905e8621 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:03.969608 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe95dc9-0b1b-4fb6-9c75-2db4905e8621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:03.971413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-efe95dc9-0b1b-4fb6-9c75-2db4905e8621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:03.985353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe95dc9-0b1b-4fb6-9c75-2db4905e8621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:03.986787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe95dc9-0b1b-4fb6-9c75-2db4905e8621 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:03.998991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1244/2492] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:05.933136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014374d7-c5bf-48c9-9ceb-d538cfc2a60e service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:05.942576 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014374d7-c5bf-48c9-9ceb-d538cfc2a60e service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:05.947238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561bc6a-f762-4823-b145-7a9899d0e81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:05.947855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561bc6a-f762-4823-b145-7a9899d0e81f service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:05.948128 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561bc6a-f762-4823-b145-7a9899d0e81f service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:05.948453 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561bc6a-f762-4823-b145-7a9899d0e81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:05.950678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0561bc6a-f762-4823-b145-7a9899d0e81f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:05.962413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561bc6a-f762-4823-b145-7a9899d0e81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:05.963838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561bc6a-f762-4823-b145-7a9899d0e81f service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:05.974034 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1249/2493] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:07.476909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe95dc9-0b1b-4fb6-9c75-2db4905e8621 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:07.486137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe95dc9-0b1b-4fb6-9c75-2db4905e8621 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:07.490748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d818db5-cb20-40bd-a3b2-0546df879679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:07.491371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d818db5-cb20-40bd-a3b2-0546df879679 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:07.491643 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d818db5-cb20-40bd-a3b2-0546df879679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:07.491914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d818db5-cb20-40bd-a3b2-0546df879679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:07.493859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7d818db5-cb20-40bd-a3b2-0546df879679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:07.561566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d818db5-cb20-40bd-a3b2-0546df879679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:07.563014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d818db5-cb20-40bd-a3b2-0546df879679 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:07.573758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1245/2494] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:10.855769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561bc6a-f762-4823-b145-7a9899d0e81f service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:10.857346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0561bc6a-f762-4823-b145-7a9899d0e81f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:10.871883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561bc6a-f762-4823-b145-7a9899d0e81f service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:10.877749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:10.878275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:10.878557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:10.878831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] PATH_INFO: `/v3/users/fc8a5fa04b6141329778221169afdfc1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:10.880109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc8a5fa04b6141329778221169afdfc1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:10.882131 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:10.934145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc8a5fa04b6141329778221169afdfc1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:10.935146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc8a5fa04b6141329778221169afdfc1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:10.945462 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc8a5fa04b6141329778221169afdfc1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:10.962168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc8a5fa04b6141329778221169afdfc1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:10.971238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-98935885-aa8d-42a2-ad18-fef2c4c79a63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc8a5fa04b6141329778221169afdfc1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:10.978021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1250/2495] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:10:10 2022] DELETE /identity/v3/users/fc8a5fa04b6141329778221169afdfc1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:11.167371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d818db5-cb20-40bd-a3b2-0546df879679 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:11.179054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d818db5-cb20-40bd-a3b2-0546df879679 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:11.238992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:11.239666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:11.240319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:11.240627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] PATH_INFO: `/v3/projects/7dc50e555f674cc7b33ac8f28b6fbd33` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:11.241903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc50e555f674cc7b33ac8f28b6fbd33)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:11.243865 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:11.291551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc50e555f674cc7b33ac8f28b6fbd33', 'request_id': 'req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:11.298755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc50e555f674cc7b33ac8f28b6fbd33', 'request_id': 'req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:11.308283 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc50e555f674cc7b33ac8f28b6fbd33', 'request_id': 'req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:11.319805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] The token cache is being invalidate because project 7dc50e555f674cc7b33ac8f28b6fbd33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:10:11.320093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:11.320756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-dc53b0dd-bd1d-41f7-9685-9c6ca51216b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:11.322326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1246/2496] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:10:11 2022] DELETE /identity/v3/projects/7dc50e555f674cc7b33ac8f28b6fbd33 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:11.327815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408715e-fde8-4e1c-8c4d-2114b0bfd26a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:11.328101 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408715e-fde8-4e1c-8c4d-2114b0bfd26a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:11.328412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408715e-fde8-4e1c-8c4d-2114b0bfd26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:11.350780 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3408715e-fde8-4e1c-8c4d-2114b0bfd26a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:11.352031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3408715e-fde8-4e1c-8c4d-2114b0bfd26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:11.404139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:11.426606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1251/2497] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:11.434830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ab675-408f-4bc5-8e29-b9563b42edb3 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:11.435043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ab675-408f-4bc5-8e29-b9563b42edb3 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:11.435323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ab675-408f-4bc5-8e29-b9563b42edb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:11.458766 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c73ab675-408f-4bc5-8e29-b9563b42edb3 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:11.460025 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c73ab675-408f-4bc5-8e29-b9563b42edb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:11.469862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1247/2498] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:11.474298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408715e-fde8-4e1c-8c4d-2114b0bfd26a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:11.484894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408715e-fde8-4e1c-8c4d-2114b0bfd26a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:11.490466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda57fc-3a9f-4442-a0c2-6cb255efd2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:11.491095 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda57fc-3a9f-4442-a0c2-6cb255efd2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:11.491386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda57fc-3a9f-4442-a0c2-6cb255efd2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:11.491654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda57fc-3a9f-4442-a0c2-6cb255efd2f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:11.492404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda57fc-3a9f-4442-a0c2-6cb255efd2f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:10:11.492696 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda57fc-3a9f-4442-a0c2-6cb255efd2f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:11.494338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda57fc-3a9f-4442-a0c2-6cb255efd2f3 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:11.506443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1252/2499] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:10:11.511019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ab675-408f-4bc5-8e29-b9563b42edb3 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:11.516794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:11.521069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ab675-408f-4bc5-8e29-b9563b42edb3 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:11.527583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc7469-7c2d-4683-9118-7c66cabd2571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:11.528882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7469-7c2d-4683-9118-7c66cabd2571 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:11.529698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7469-7c2d-4683-9118-7c66cabd2571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:11.530017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc7469-7c2d-4683-9118-7c66cabd2571 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:11.530786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc7469-7c2d-4683-9118-7c66cabd2571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:11.532950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc7469-7c2d-4683-9118-7c66cabd2571 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:11.566248 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1248/2500] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:10:11 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:11.571634 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda57fc-3a9f-4442-a0c2-6cb255efd2f3 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:11.583027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda57fc-3a9f-4442-a0c2-6cb255efd2f3 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:11.648993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3048bdb-52a1-48a1-8622-9c4de9f7e720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:11.649631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3048bdb-52a1-48a1-8622-9c4de9f7e720 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:11.649908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3048bdb-52a1-48a1-8622-9c4de9f7e720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:11.650184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3048bdb-52a1-48a1-8622-9c4de9f7e720 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:11.650853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3048bdb-52a1-48a1-8622-9c4de9f7e720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:11.652835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3048bdb-52a1-48a1-8622-9c4de9f7e720 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:11.675753 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1253/2501] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:10:11 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:11.680885 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc7469-7c2d-4683-9118-7c66cabd2571 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:11.692262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc7469-7c2d-4683-9118-7c66cabd2571 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:11.697662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435d377-b0ac-430f-81cb-8e0100756b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:11.698287 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435d377-b0ac-430f-81cb-8e0100756b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:11.698544 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435d377-b0ac-430f-81cb-8e0100756b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:11.698828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435d377-b0ac-430f-81cb-8e0100756b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:11.699525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435d377-b0ac-430f-81cb-8e0100756b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:11.701173 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435d377-b0ac-430f-81cb-8e0100756b2b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:11.711738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1249/2502] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:11.716110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3048bdb-52a1-48a1-8622-9c4de9f7e720 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:11.726434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3048bdb-52a1-48a1-8622-9c4de9f7e720 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:11.731850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030511e-0b2a-49cd-832e-80b25071588d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:11.732490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030511e-0b2a-49cd-832e-80b25071588d admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:11.732906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030511e-0b2a-49cd-832e-80b25071588d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:11.733182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030511e-0b2a-49cd-832e-80b25071588d admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:11.733885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030511e-0b2a-49cd-832e-80b25071588d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:11.735514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030511e-0b2a-49cd-832e-80b25071588d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:11.749467 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1254/2503] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:11.754222 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435d377-b0ac-430f-81cb-8e0100756b2b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:11.767933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435d377-b0ac-430f-81cb-8e0100756b2b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:11.773148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a93472-9c30-42e9-98fc-ab8a512b4473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:11.773816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a93472-9c30-42e9-98fc-ab8a512b4473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:11.774104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a93472-9c30-42e9-98fc-ab8a512b4473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:11.774349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a93472-9c30-42e9-98fc-ab8a512b4473 admin admin] PATH_INFO: `/v3/projects/8181e02cb490487295d1251fe5e95087/users/ec5e21f09dd24244afdd7fe6158930f9/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:11.790491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a93472-9c30-42e9-98fc-ab8a512b4473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8181e02cb490487295d1251fe5e95087, user_id=ec5e21f09dd24244afdd7fe6158930f9, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:11.794786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a93472-9c30-42e9-98fc-ab8a512b4473 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:11.812021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1250/2504] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:11 2022] PUT /identity/v3/projects/8181e02cb490487295d1251fe5e95087/users/ec5e21f09dd24244afdd7fe6158930f9/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:11.826669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47669b2-868a-44ab-b4d0-fef040736326 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:11.826977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47669b2-868a-44ab-b4d0-fef040736326 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:11.827304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47669b2-868a-44ab-b4d0-fef040736326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:11.856750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d47669b2-868a-44ab-b4d0-fef040736326 None None] MFA Rules not processed for user `ec5e21f09dd24244afdd7fe6158930f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:11.858133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d47669b2-868a-44ab-b4d0-fef040736326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:11.923493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1255/2505] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:11.938418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a93472-9c30-42e9-98fc-ab8a512b4473 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:11.940282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-29a93472-9c30-42e9-98fc-ab8a512b4473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:12.028994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a93472-9c30-42e9-98fc-ab8a512b4473 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:12.035117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb276dc-336e-43da-9cac-abb99af65345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:12.035746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb276dc-336e-43da-9cac-abb99af65345 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:12.036028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb276dc-336e-43da-9cac-abb99af65345 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:12.036322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb276dc-336e-43da-9cac-abb99af65345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:12.044282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb276dc-336e-43da-9cac-abb99af65345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:12.046069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb276dc-336e-43da-9cac-abb99af65345 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:12.054955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1251/2506] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:14.797740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47669b2-868a-44ab-b4d0-fef040736326 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:14.842235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:14.864244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47669b2-868a-44ab-b4d0-fef040736326 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:14.869653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:14.870303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:14.870579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:14.870838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] PATH_INFO: `/v3/users/ec5e21f09dd24244afdd7fe6158930f9` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:14.872180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5e21f09dd24244afdd7fe6158930f9)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:14.874201 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:14.928247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5e21f09dd24244afdd7fe6158930f9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:14.929070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5e21f09dd24244afdd7fe6158930f9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:14.937855 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5e21f09dd24244afdd7fe6158930f9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:14.950588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5e21f09dd24244afdd7fe6158930f9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:14.957447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5e21f09dd24244afdd7fe6158930f9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:14.963440 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1256/2507] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:10:14 2022] DELETE /identity/v3/users/ec5e21f09dd24244afdd7fe6158930f9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:14.969436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bc3b1-c4f3-421f-82ad-cbd03656420e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:14.969930 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bc3b1-c4f3-421f-82ad-cbd03656420e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:14.970026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23bc3b1-c4f3-421f-82ad-cbd03656420e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:14.990566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b23bc3b1-c4f3-421f-82ad-cbd03656420e None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:14.991762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b23bc3b1-c4f3-421f-82ad-cbd03656420e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:15.055352 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1252/2508] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:15.065764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:15.067550 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:15.132517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5673e06-625c-4b38-ada5-91edc177c633 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:15.136832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68d109-b713-4bc9-9744-7232ad9b529e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:15.137423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d109-b713-4bc9-9744-7232ad9b529e service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:15.137700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d109-b713-4bc9-9744-7232ad9b529e service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:15.137992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68d109-b713-4bc9-9744-7232ad9b529e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:15.148793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68d109-b713-4bc9-9744-7232ad9b529e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:15.150233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68d109-b713-4bc9-9744-7232ad9b529e service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:15.160891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1257/2509] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:15.456639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bc3b1-c4f3-421f-82ad-cbd03656420e None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:15.458148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b23bc3b1-c4f3-421f-82ad-cbd03656420e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:15.473284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23bc3b1-c4f3-421f-82ad-cbd03656420e None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:15.479663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:15.480516 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:15.480829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:15.481073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] PATH_INFO: `/v3/projects/8181e02cb490487295d1251fe5e95087` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:15.482334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8181e02cb490487295d1251fe5e95087)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:15.484423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:15.519108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:15.535109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8181e02cb490487295d1251fe5e95087', 'request_id': 'req-6af51240-2a9b-4915-9d67-a5fe636fff6f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:15.541409 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8181e02cb490487295d1251fe5e95087', 'request_id': 'req-6af51240-2a9b-4915-9d67-a5fe636fff6f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:15.550064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8181e02cb490487295d1251fe5e95087', 'request_id': 'req-6af51240-2a9b-4915-9d67-a5fe636fff6f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:15.561600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] The token cache is being invalidate because project 8181e02cb490487295d1251fe5e95087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:10:15.561916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:15.562720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6af51240-2a9b-4915-9d67-a5fe636fff6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:15.564342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1253/2510] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:10:15 2022] DELETE /identity/v3/projects/8181e02cb490487295d1251fe5e95087 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:15.569720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8a512-868f-40ad-b95c-831606c34737 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:15.569950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8a512-868f-40ad-b95c-831606c34737 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:15.570222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8a512-868f-40ad-b95c-831606c34737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:15.592221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-ced8a512-868f-40ad-b95c-831606c34737 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:15.593521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ced8a512-868f-40ad-b95c-831606c34737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:15.656801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1258/2511] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:15.668945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41886c-273e-4e1c-9924-df5cd9605f96 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:15.669328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41886c-273e-4e1c-9924-df5cd9605f96 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:15.669696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d41886c-273e-4e1c-9924-df5cd9605f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:15.694869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-5d41886c-273e-4e1c-9924-df5cd9605f96 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:15.696604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5d41886c-273e-4e1c-9924-df5cd9605f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:15.710512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1254/2512] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:15.716329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8a512-868f-40ad-b95c-831606c34737 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:15.723715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:15.730295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8a512-868f-40ad-b95c-831606c34737 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:15.738810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326eaa2-3715-4c20-acad-d21b537fd7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:15.739450 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326eaa2-3715-4c20-acad-d21b537fd7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:15.739735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326eaa2-3715-4c20-acad-d21b537fd7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:15.740434 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326eaa2-3715-4c20-acad-d21b537fd7e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:15.741151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326eaa2-3715-4c20-acad-d21b537fd7e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:10:15.741469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326eaa2-3715-4c20-acad-d21b537fd7e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:15.743773 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326eaa2-3715-4c20-acad-d21b537fd7e9 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:15.756910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1259/2513] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:10:15.763252 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41886c-273e-4e1c-9924-df5cd9605f96 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:15.773557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d41886c-273e-4e1c-9924-df5cd9605f96 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:15.778602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d4376-4dd0-4de4-88a6-a640892547e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:15.779217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d4376-4dd0-4de4-88a6-a640892547e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:15.779501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d4376-4dd0-4de4-88a6-a640892547e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:15.779762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d4376-4dd0-4de4-88a6-a640892547e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:15.780477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d4376-4dd0-4de4-88a6-a640892547e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:15.782436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d4376-4dd0-4de4-88a6-a640892547e7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:15.815058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1255/2514] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:10:15 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:10:15.819614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326eaa2-3715-4c20-acad-d21b537fd7e9 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:15.830603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326eaa2-3715-4c20-acad-d21b537fd7e9 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:15.888466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79719d-9654-427f-bd96-a13f2e0e3019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:15.889080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79719d-9654-427f-bd96-a13f2e0e3019 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:15.889356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79719d-9654-427f-bd96-a13f2e0e3019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:15.889624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d79719d-9654-427f-bd96-a13f2e0e3019 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:15.890310 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79719d-9654-427f-bd96-a13f2e0e3019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:15.892224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d79719d-9654-427f-bd96-a13f2e0e3019 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:15.920046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1260/2515] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:10:15 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:15.924627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d4376-4dd0-4de4-88a6-a640892547e7 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:15.936358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d4376-4dd0-4de4-88a6-a640892547e7 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:15.942164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819a0a5-fce8-4570-b6d8-666db0fd74d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:15.942796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819a0a5-fce8-4570-b6d8-666db0fd74d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:15.943071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819a0a5-fce8-4570-b6d8-666db0fd74d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:15.943341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819a0a5-fce8-4570-b6d8-666db0fd74d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:15.944040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819a0a5-fce8-4570-b6d8-666db0fd74d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:15.945744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819a0a5-fce8-4570-b6d8-666db0fd74d7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:16.088333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1256/2516] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:16.092686 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79719d-9654-427f-bd96-a13f2e0e3019 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:16.103946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d79719d-9654-427f-bd96-a13f2e0e3019 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:16.109846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1ebab-64ec-4faf-a852-4f90d8cf1eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:16.110477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1ebab-64ec-4faf-a852-4f90d8cf1eae admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.110756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1ebab-64ec-4faf-a852-4f90d8cf1eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.111026 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1ebab-64ec-4faf-a852-4f90d8cf1eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.111760 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1ebab-64ec-4faf-a852-4f90d8cf1eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:16.113518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1ebab-64ec-4faf-a852-4f90d8cf1eae admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:16.124965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1261/2517] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:16.129492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819a0a5-fce8-4570-b6d8-666db0fd74d7 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:16.141862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819a0a5-fce8-4570-b6d8-666db0fd74d7 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:16.147640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fc82e-a802-4c31-a41b-14b5014c536b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:16.148365 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fc82e-a802-4c31-a41b-14b5014c536b admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.148639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fc82e-a802-4c31-a41b-14b5014c536b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.148905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55fc82e-a802-4c31-a41b-14b5014c536b admin admin] PATH_INFO: `/v3/projects/a17b3c27a02c4e928067350596188751/users/9f17058c05354dd982cd2e53a61457f8/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.167426 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fc82e-a802-4c31-a41b-14b5014c536b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17b3c27a02c4e928067350596188751, user_id=9f17058c05354dd982cd2e53a61457f8, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:16.170135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55fc82e-a802-4c31-a41b-14b5014c536b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:16.191829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1257/2518] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:16 2022] PUT /identity/v3/projects/a17b3c27a02c4e928067350596188751/users/9f17058c05354dd982cd2e53a61457f8/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:16.202426 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b839e-f2d4-42bb-8fc5-62f54aff6a3c None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.202716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b839e-f2d4-42bb-8fc5-62f54aff6a3c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.203180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b839e-f2d4-42bb-8fc5-62f54aff6a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.234908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-f15b839e-f2d4-42bb-8fc5-62f54aff6a3c None None] MFA Rules not processed for user `9f17058c05354dd982cd2e53a61457f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:16.236127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f15b839e-f2d4-42bb-8fc5-62f54aff6a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:16.311994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1262/2519] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:16.313358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fc82e-a802-4c31-a41b-14b5014c536b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:16.380059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55fc82e-a802-4c31-a41b-14b5014c536b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:16.385618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b153e9-3ecd-4cd3-b5bf-3240dff93215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:16.386047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b153e9-3ecd-4cd3-b5bf-3240dff93215 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.386354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b153e9-3ecd-4cd3-b5bf-3240dff93215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.386622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b153e9-3ecd-4cd3-b5bf-3240dff93215 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.387288 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b153e9-3ecd-4cd3-b5bf-3240dff93215 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:16.389310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b153e9-3ecd-4cd3-b5bf-3240dff93215 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:16.422370 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1258/2520] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:10:16 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:16.426562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b839e-f2d4-42bb-8fc5-62f54aff6a3c None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:16.437592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b839e-f2d4-42bb-8fc5-62f54aff6a3c None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:16.490354 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:16.496873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268bcfd-0d6a-482f-9832-6985b7fcaecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:16.497596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268bcfd-0d6a-482f-9832-6985b7fcaecc admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.498052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268bcfd-0d6a-482f-9832-6985b7fcaecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.498487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f268bcfd-0d6a-482f-9832-6985b7fcaecc admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.499810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268bcfd-0d6a-482f-9832-6985b7fcaecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:16.501886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f268bcfd-0d6a-482f-9832-6985b7fcaecc admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:16.525033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1263/2521] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:10:16 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:10:16.529049 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b153e9-3ecd-4cd3-b5bf-3240dff93215 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:16.540473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b153e9-3ecd-4cd3-b5bf-3240dff93215 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:16.545995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a6752-a9a7-4123-ae4e-94f23e3ec2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:16.546673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a6752-a9a7-4123-ae4e-94f23e3ec2cb admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.546953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a6752-a9a7-4123-ae4e-94f23e3ec2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.547221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a6752-a9a7-4123-ae4e-94f23e3ec2cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.547920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a6752-a9a7-4123-ae4e-94f23e3ec2cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:16.549616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a6752-a9a7-4123-ae4e-94f23e3ec2cb admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:16.560226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1259/2522] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:16.564925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268bcfd-0d6a-482f-9832-6985b7fcaecc admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:16.575016 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f268bcfd-0d6a-482f-9832-6985b7fcaecc admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:16.579657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce0fd1-3ea9-49ed-911b-033998d4054b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:16.580349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce0fd1-3ea9-49ed-911b-033998d4054b admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.580632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce0fd1-3ea9-49ed-911b-033998d4054b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.580908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce0fd1-3ea9-49ed-911b-033998d4054b admin admin] PATH_INFO: `/v3/projects/77d6b6f039334f03870cd79daa19a714/users/597ae49fdd8c4b0b8a900d44ca56d473/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.595983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce0fd1-3ea9-49ed-911b-033998d4054b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d6b6f039334f03870cd79daa19a714, user_id=597ae49fdd8c4b0b8a900d44ca56d473, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:16.598710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce0fd1-3ea9-49ed-911b-033998d4054b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:16.615791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1264/2523] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:16 2022] PUT /identity/v3/projects/77d6b6f039334f03870cd79daa19a714/users/597ae49fdd8c4b0b8a900d44ca56d473/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:16.620124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a6752-a9a7-4123-ae4e-94f23e3ec2cb admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:16.630989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a6752-a9a7-4123-ae4e-94f23e3ec2cb admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:16.657212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:16.694062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0de8c-f9bd-48b2-a7b6-9952c6967cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:16.694684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0de8c-f9bd-48b2-a7b6-9952c6967cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.694954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0de8c-f9bd-48b2-a7b6-9952c6967cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.695217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0de8c-f9bd-48b2-a7b6-9952c6967cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.695917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0de8c-f9bd-48b2-a7b6-9952c6967cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:16.697572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0de8c-f9bd-48b2-a7b6-9952c6967cd1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:16.707821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1260/2524] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:10:16.712217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce0fd1-3ea9-49ed-911b-033998d4054b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:16.723235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce0fd1-3ea9-49ed-911b-033998d4054b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:16.729824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b8351-49eb-4607-9487-b887318538c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:16.730791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8351-49eb-4607-9487-b887318538c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.731174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8351-49eb-4607-9487-b887318538c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.731694 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b8351-49eb-4607-9487-b887318538c6 admin admin] PATH_INFO: `/v3/projects/77d6b6f039334f03870cd79daa19a714/users/597ae49fdd8c4b0b8a900d44ca56d473/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.735103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b8351-49eb-4607-9487-b887318538c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d6b6f039334f03870cd79daa19a714, user_id=597ae49fdd8c4b0b8a900d44ca56d473, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:16.739350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b8351-49eb-4607-9487-b887318538c6 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:16.754124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1265/2525] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:16 2022] PUT /identity/v3/projects/77d6b6f039334f03870cd79daa19a714/users/597ae49fdd8c4b0b8a900d44ca56d473/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:16.766310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9b498-ed1e-4c3d-913a-4db486f8ebed None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.766595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9b498-ed1e-4c3d-913a-4db486f8ebed None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.766876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9b498-ed1e-4c3d-913a-4db486f8ebed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.795901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1bd9b498-ed1e-4c3d-913a-4db486f8ebed None None] MFA Rules not processed for user `597ae49fdd8c4b0b8a900d44ca56d473`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:16.797276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1bd9b498-ed1e-4c3d-913a-4db486f8ebed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:16.871878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1261/2526] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:16.883940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b8351-49eb-4607-9487-b887318538c6 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:16.885968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e04b8351-49eb-4607-9487-b887318538c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:16.945593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b8351-49eb-4607-9487-b887318538c6 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:16.950054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028cc5c-fa9e-4657-a119-a6ddaaf5cf72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:16.950678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028cc5c-fa9e-4657-a119-a6ddaaf5cf72 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:16.950962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028cc5c-fa9e-4657-a119-a6ddaaf5cf72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:16.951237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028cc5c-fa9e-4657-a119-a6ddaaf5cf72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:16.960787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028cc5c-fa9e-4657-a119-a6ddaaf5cf72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:16.962232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028cc5c-fa9e-4657-a119-a6ddaaf5cf72 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:16.971772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1266/2527] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:17.050892 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9b498-ed1e-4c3d-913a-4db486f8ebed None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:17.121391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9b498-ed1e-4c3d-913a-4db486f8ebed None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:17.126242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:17.126834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:17.127115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:17.127383 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] PATH_INFO: `/v3/users/9f17058c05354dd982cd2e53a61457f8` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:17.128604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f17058c05354dd982cd2e53a61457f8)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:17.130537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:17.191093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f17058c05354dd982cd2e53a61457f8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.193653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:17.206979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f17058c05354dd982cd2e53a61457f8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.215023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f17058c05354dd982cd2e53a61457f8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.215867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f17058c05354dd982cd2e53a61457f8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.216553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f17058c05354dd982cd2e53a61457f8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.230209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1262/2528] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:10:17 2022] DELETE /identity/v3/users/9f17058c05354dd982cd2e53a61457f8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:17.235952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fd0cb-1828-4df8-afa9-961de4126380 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:17.236349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fd0cb-1828-4df8-afa9-961de4126380 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:17.236765 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029fd0cb-1828-4df8-afa9-961de4126380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:17.264108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-029fd0cb-1828-4df8-afa9-961de4126380 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:17.265683 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-029fd0cb-1828-4df8-afa9-961de4126380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:17.331993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1267/2529] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:17.342263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:17.343966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:17.412446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74aa85-7923-49f0-b4e1-910f3a8a4539 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:17.416677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05ba89-143a-4ae0-a46e-bd0d73e315e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:17.417309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ba89-143a-4ae0-a46e-bd0d73e315e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:17.417579 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ba89-143a-4ae0-a46e-bd0d73e315e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:17.417861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ba89-143a-4ae0-a46e-bd0d73e315e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:17.428105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05ba89-143a-4ae0-a46e-bd0d73e315e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:17.429585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05ba89-143a-4ae0-a46e-bd0d73e315e1 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:17.439800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1263/2530] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:17.725797 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fd0cb-1828-4df8-afa9-961de4126380 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:17.727342 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-029fd0cb-1828-4df8-afa9-961de4126380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:17.741251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029fd0cb-1828-4df8-afa9-961de4126380 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:17.751597 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:17.752278 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:17.752590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:17.752915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] PATH_INFO: `/v3/projects/a17b3c27a02c4e928067350596188751` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:17.754101 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a17b3c27a02c4e928067350596188751)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:17.758323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:17.806204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:17.807575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a17b3c27a02c4e928067350596188751', 'request_id': 'req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.815364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a17b3c27a02c4e928067350596188751', 'request_id': 'req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.825750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a17b3c27a02c4e928067350596188751', 'request_id': 'req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.837294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] The token cache is being invalidate because project a17b3c27a02c4e928067350596188751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:10:17.837589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.838242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.839751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1268/2531] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:10:17 2022] DELETE /identity/v3/projects/a17b3c27a02c4e928067350596188751 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:17.844186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05ba89-143a-4ae0-a46e-bd0d73e315e1 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:17.854584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05ba89-143a-4ae0-a46e-bd0d73e315e1 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:17.916619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:17.917269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:17.917677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:17.917955 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] PATH_INFO: `/v3/users/597ae49fdd8c4b0b8a900d44ca56d473` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:17.919231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=597ae49fdd8c4b0b8a900d44ca56d473)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:17.921155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:17.971378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '597ae49fdd8c4b0b8a900d44ca56d473'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.984502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '597ae49fdd8c4b0b8a900d44ca56d473'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.990322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:17.992444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '597ae49fdd8c4b0b8a900d44ca56d473'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.993359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '597ae49fdd8c4b0b8a900d44ca56d473'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:17.994081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-54a1956e-769c-416d-88c7-56ead2c22ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '597ae49fdd8c4b0b8a900d44ca56d473'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:18.008791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1264/2532] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:10:17 2022] DELETE /identity/v3/users/597ae49fdd8c4b0b8a900d44ca56d473 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:18.321058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:18.322603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:18.336941 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce79dc7-4701-4c98-bac9-7b1f70b720f2 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:18.398907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:18.399591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:18.399872 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:18.400146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] PATH_INFO: `/v3/projects/77d6b6f039334f03870cd79daa19a714` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:18.401755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d6b6f039334f03870cd79daa19a714)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:18.403684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:18.474456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d6b6f039334f03870cd79daa19a714', 'request_id': 'req-e14acff7-0ff9-4206-9539-fd307b5706e7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:18.481460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d6b6f039334f03870cd79daa19a714', 'request_id': 'req-e14acff7-0ff9-4206-9539-fd307b5706e7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:18.494999 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d6b6f039334f03870cd79daa19a714', 'request_id': 'req-e14acff7-0ff9-4206-9539-fd307b5706e7'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:18.507317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] The token cache is being invalidate because project 77d6b6f039334f03870cd79daa19a714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:10:18.507691 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:18.508432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e14acff7-0ff9-4206-9539-fd307b5706e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:18.510048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1269/2533] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:10:18 2022] DELETE /identity/v3/projects/77d6b6f039334f03870cd79daa19a714 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:18.515424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f477f-4870-4202-ad51-178aed5f90e2 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:18.515904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f477f-4870-4202-ad51-178aed5f90e2 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:18.515991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f477f-4870-4202-ad51-178aed5f90e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:18.535811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c22f477f-4870-4202-ad51-178aed5f90e2 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:18.537057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c22f477f-4870-4202-ad51-178aed5f90e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:18.597434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1265/2534] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:18.606435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f1d5-eb32-4e61-a628-e8d9e79696b1 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:18.606876 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f1d5-eb32-4e61-a628-e8d9e79696b1 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:18.607321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f1d5-eb32-4e61-a628-e8d9e79696b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:18.638700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6353f1d5-eb32-4e61-a628-e8d9e79696b1 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:18.640645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6353f1d5-eb32-4e61-a628-e8d9e79696b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:18.650745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1270/2535] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:18.656109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f477f-4870-4202-ad51-178aed5f90e2 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:18.669389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f477f-4870-4202-ad51-178aed5f90e2 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:18.676935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a9f6c-f578-4b4e-9d7c-e0f314f7e2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:18.677742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9f6c-f578-4b4e-9d7c-e0f314f7e2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:18.678109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9f6c-f578-4b4e-9d7c-e0f314f7e2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:18.678418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9f6c-f578-4b4e-9d7c-e0f314f7e2e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:18.679429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a9f6c-f578-4b4e-9d7c-e0f314f7e2e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:10:18.679871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a9f6c-f578-4b4e-9d7c-e0f314f7e2e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:18.682088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a9f6c-f578-4b4e-9d7c-e0f314f7e2e5 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:18.699809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1266/2536] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:10:18.704645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353f1d5-eb32-4e61-a628-e8d9e79696b1 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:18.714396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353f1d5-eb32-4e61-a628-e8d9e79696b1 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:18.719577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa850c6c-5c6a-4392-a538-2f3560344466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:18.720184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa850c6c-5c6a-4392-a538-2f3560344466 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:18.720496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa850c6c-5c6a-4392-a538-2f3560344466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:18.720768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa850c6c-5c6a-4392-a538-2f3560344466 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:18.721441 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa850c6c-5c6a-4392-a538-2f3560344466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:18.723412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa850c6c-5c6a-4392-a538-2f3560344466 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:18.755120 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1271/2537] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:10:18 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:18.760088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a9f6c-f578-4b4e-9d7c-e0f314f7e2e5 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:18.771936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a9f6c-f578-4b4e-9d7c-e0f314f7e2e5 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:18.838850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b28db3-a53c-49ac-89ca-35ccd9b567ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:18.839481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b28db3-a53c-49ac-89ca-35ccd9b567ce admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:18.839792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b28db3-a53c-49ac-89ca-35ccd9b567ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:18.840051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b28db3-a53c-49ac-89ca-35ccd9b567ce admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:18.840773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b28db3-a53c-49ac-89ca-35ccd9b567ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:18.842812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b28db3-a53c-49ac-89ca-35ccd9b567ce admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:18.868180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1267/2538] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:10:18 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:10:18.872238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa850c6c-5c6a-4392-a538-2f3560344466 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:18.883045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa850c6c-5c6a-4392-a538-2f3560344466 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:18.888110 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640e475-1c50-4e72-add6-a4e1e096a465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:18.888770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640e475-1c50-4e72-add6-a4e1e096a465 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:18.889130 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640e475-1c50-4e72-add6-a4e1e096a465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:18.889315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9640e475-1c50-4e72-add6-a4e1e096a465 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:18.890019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640e475-1c50-4e72-add6-a4e1e096a465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:18.891669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9640e475-1c50-4e72-add6-a4e1e096a465 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:18.905733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1272/2539] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:18.907325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b28db3-a53c-49ac-89ca-35ccd9b567ce admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:18.917592 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b28db3-a53c-49ac-89ca-35ccd9b567ce admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:18.922538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93f5a54-f2d2-4bb9-856c-4c69d36bd1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:18.923158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f5a54-f2d2-4bb9-856c-4c69d36bd1ed admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:18.923438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f5a54-f2d2-4bb9-856c-4c69d36bd1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:18.923712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93f5a54-f2d2-4bb9-856c-4c69d36bd1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:18.924452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f5a54-f2d2-4bb9-856c-4c69d36bd1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:18.926061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93f5a54-f2d2-4bb9-856c-4c69d36bd1ed admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:18.937542 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1268/2540] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:18.943868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640e475-1c50-4e72-add6-a4e1e096a465 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:18.961203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9640e475-1c50-4e72-add6-a4e1e096a465 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:18.969603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f57da-8545-4fe1-b274-4e5dfe75a928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:18.970272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f57da-8545-4fe1-b274-4e5dfe75a928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:18.970549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f57da-8545-4fe1-b274-4e5dfe75a928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:18.970822 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f57da-8545-4fe1-b274-4e5dfe75a928 admin admin] PATH_INFO: `/v3/projects/edcdd3a879c343d2bf87c56837522adc/users/2f49d6a07d5d422696bde651262de83e/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:18.986887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f57da-8545-4fe1-b274-4e5dfe75a928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edcdd3a879c343d2bf87c56837522adc, user_id=2f49d6a07d5d422696bde651262de83e, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:18.989538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392f57da-8545-4fe1-b274-4e5dfe75a928 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:19.004539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1273/2541] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:18 2022] PUT /identity/v3/projects/edcdd3a879c343d2bf87c56837522adc/users/2f49d6a07d5d422696bde651262de83e/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:19.015097 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eee57c-fd11-4945-8319-d125c4e4fcd5 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:19.015375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eee57c-fd11-4945-8319-d125c4e4fcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:19.015639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eee57c-fd11-4945-8319-d125c4e4fcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:19.045207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-19eee57c-fd11-4945-8319-d125c4e4fcd5 None None] MFA Rules not processed for user `2f49d6a07d5d422696bde651262de83e`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:19.046509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-19eee57c-fd11-4945-8319-d125c4e4fcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:19.100476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:19.126307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1269/2542] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:19.131025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f57da-8545-4fe1-b274-4e5dfe75a928 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:19.136118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:19.209976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f57da-8545-4fe1-b274-4e5dfe75a928 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:19.215349 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbcb2b-7b56-4567-9d94-13c975c3f27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:19.215972 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbcb2b-7b56-4567-9d94-13c975c3f27f admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:19.216246 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbcb2b-7b56-4567-9d94-13c975c3f27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:19.216554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbcb2b-7b56-4567-9d94-13c975c3f27f admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:19.217218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbcb2b-7b56-4567-9d94-13c975c3f27f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:19.219153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbcb2b-7b56-4567-9d94-13c975c3f27f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:19.248619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1274/2543] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:10:19 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:19.253138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eee57c-fd11-4945-8319-d125c4e4fcd5 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:19.264411 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eee57c-fd11-4945-8319-d125c4e4fcd5 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:19.331660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378e52d-8a2b-4ef8-b60f-2e4194776e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:19.332306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378e52d-8a2b-4ef8-b60f-2e4194776e9f admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:19.332629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378e52d-8a2b-4ef8-b60f-2e4194776e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:19.332875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378e52d-8a2b-4ef8-b60f-2e4194776e9f admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:19.333519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378e52d-8a2b-4ef8-b60f-2e4194776e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:19.335487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378e52d-8a2b-4ef8-b60f-2e4194776e9f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:19.361061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1270/2544] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:10:19 2022] POST /identity/v3/users => generated 298 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:10:19.365692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbcb2b-7b56-4567-9d94-13c975c3f27f admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:19.376587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbcb2b-7b56-4567-9d94-13c975c3f27f admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:19.381940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1fd1c-edad-4d1d-9add-5f4cb10f19aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:19.382553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1fd1c-edad-4d1d-9add-5f4cb10f19aa admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:19.382830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1fd1c-edad-4d1d-9add-5f4cb10f19aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:19.383097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1fd1c-edad-4d1d-9add-5f4cb10f19aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:19.383793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1fd1c-edad-4d1d-9add-5f4cb10f19aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:19.385452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1fd1c-edad-4d1d-9add-5f4cb10f19aa admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:19.396802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1275/2545] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:19.401068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378e52d-8a2b-4ef8-b60f-2e4194776e9f admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:19.411715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378e52d-8a2b-4ef8-b60f-2e4194776e9f admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:19.417742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbff61b-0fbf-42f4-9ed6-c9053790e55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:19.418391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbff61b-0fbf-42f4-9ed6-c9053790e55a admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:19.418666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbff61b-0fbf-42f4-9ed6-c9053790e55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:19.418963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbff61b-0fbf-42f4-9ed6-c9053790e55a admin admin] PATH_INFO: `/v3/projects/c31ad0b96be84abbb0f17cd81646b563/users/2151ef01cd144ccc8a80b81f0ba9efd8/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:19.434282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbff61b-0fbf-42f4-9ed6-c9053790e55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31ad0b96be84abbb0f17cd81646b563, user_id=2151ef01cd144ccc8a80b81f0ba9efd8, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:19.437016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbff61b-0fbf-42f4-9ed6-c9053790e55a admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:19.451698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1271/2546] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:19 2022] PUT /identity/v3/projects/c31ad0b96be84abbb0f17cd81646b563/users/2151ef01cd144ccc8a80b81f0ba9efd8/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:10:19.455997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1fd1c-edad-4d1d-9add-5f4cb10f19aa admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:19.466712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1fd1c-edad-4d1d-9add-5f4cb10f19aa admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:19.530799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeb618-14d8-43a1-954b-2d4638d61f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:19.531383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb618-14d8-43a1-954b-2d4638d61f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:19.531666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb618-14d8-43a1-954b-2d4638d61f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:19.531946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeb618-14d8-43a1-954b-2d4638d61f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:19.532827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeb618-14d8-43a1-954b-2d4638d61f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:19.534456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeb618-14d8-43a1-954b-2d4638d61f73 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:19.545576 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1276/2547] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:19.552652 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbff61b-0fbf-42f4-9ed6-c9053790e55a admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:19.563907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbff61b-0fbf-42f4-9ed6-c9053790e55a admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:19.568992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b9bc7-537b-44b1-aada-f4e33cabb8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:19.569660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b9bc7-537b-44b1-aada-f4e33cabb8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:19.569940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b9bc7-537b-44b1-aada-f4e33cabb8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:19.570197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b9bc7-537b-44b1-aada-f4e33cabb8f1 admin admin] PATH_INFO: `/v3/projects/c31ad0b96be84abbb0f17cd81646b563/users/2151ef01cd144ccc8a80b81f0ba9efd8/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:19.572329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b9bc7-537b-44b1-aada-f4e33cabb8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31ad0b96be84abbb0f17cd81646b563, user_id=2151ef01cd144ccc8a80b81f0ba9efd8, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:19.575035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b9bc7-537b-44b1-aada-f4e33cabb8f1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:19.586862 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1272/2548] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:19 2022] PUT /identity/v3/projects/c31ad0b96be84abbb0f17cd81646b563/users/2151ef01cd144ccc8a80b81f0ba9efd8/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:19.598731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c326e6-9711-4394-acea-2b01b050d3fb None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:19.599015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c326e6-9711-4394-acea-2b01b050d3fb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:19.599287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c326e6-9711-4394-acea-2b01b050d3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:19.637716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-15c326e6-9711-4394-acea-2b01b050d3fb None None] MFA Rules not processed for user `2151ef01cd144ccc8a80b81f0ba9efd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:19.638935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-15c326e6-9711-4394-acea-2b01b050d3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:19.703526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:19.710408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1277/2549] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:19.721305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b9bc7-537b-44b1-aada-f4e33cabb8f1 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:19.723133 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-198b9bc7-537b-44b1-aada-f4e33cabb8f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:19.791022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b9bc7-537b-44b1-aada-f4e33cabb8f1 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:19.795047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d42bb3-dfe1-44fa-8740-728057b5547f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:19.795779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d42bb3-dfe1-44fa-8740-728057b5547f service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:19.796028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d42bb3-dfe1-44fa-8740-728057b5547f service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:19.796367 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d42bb3-dfe1-44fa-8740-728057b5547f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:19.806944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d42bb3-dfe1-44fa-8740-728057b5547f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:19.808423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d42bb3-dfe1-44fa-8740-728057b5547f service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:19.818225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1273/2550] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:19.925067 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c326e6-9711-4394-acea-2b01b050d3fb None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:19.935671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c326e6-9711-4394-acea-2b01b050d3fb None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:19.939936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe55002-1456-4261-8a0b-c17eb7eef106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:19.940602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe55002-1456-4261-8a0b-c17eb7eef106 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:19.940875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe55002-1456-4261-8a0b-c17eb7eef106 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:19.941142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe55002-1456-4261-8a0b-c17eb7eef106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:20.000413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe55002-1456-4261-8a0b-c17eb7eef106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:20.001906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe55002-1456-4261-8a0b-c17eb7eef106 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:20.011136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1278/2551] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:22.634499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d42bb3-dfe1-44fa-8740-728057b5547f service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:22.682732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:22.704831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d42bb3-dfe1-44fa-8740-728057b5547f service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:22.709884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:22.710531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:22.710829 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:22.711114 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] PATH_INFO: `/v3/users/2f49d6a07d5d422696bde651262de83e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:22.712326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f49d6a07d5d422696bde651262de83e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:22.714275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:22.771218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f49d6a07d5d422696bde651262de83e'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:22.781434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f49d6a07d5d422696bde651262de83e'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:22.789253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f49d6a07d5d422696bde651262de83e'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:22.789983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f49d6a07d5d422696bde651262de83e'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:22.790589 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f49d6a07d5d422696bde651262de83e'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:22.806294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1274/2552] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:10:22 2022] DELETE /identity/v3/users/2f49d6a07d5d422696bde651262de83e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:22.811107 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c878e90-aa41-45f3-9258-5e44d218dba6 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:22.811603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c878e90-aa41-45f3-9258-5e44d218dba6 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:22.811719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c878e90-aa41-45f3-9258-5e44d218dba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:22.835371 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9c878e90-aa41-45f3-9258-5e44d218dba6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:22.836623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9c878e90-aa41-45f3-9258-5e44d218dba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:22.906642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1279/2553] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:22.923928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:22.925665 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:22.998946 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dabbd-94a8-4394-be33-c362fd2dfaec admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:23.002907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732091de-b048-4b05-bf3b-65d5fffd661d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:23.003567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732091de-b048-4b05-bf3b-65d5fffd661d service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:23.003845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732091de-b048-4b05-bf3b-65d5fffd661d service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:23.004115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732091de-b048-4b05-bf3b-65d5fffd661d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:23.014351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732091de-b048-4b05-bf3b-65d5fffd661d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:23.015805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732091de-b048-4b05-bf3b-65d5fffd661d service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:23.025871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1275/2554] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:23.320072 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c878e90-aa41-45f3-9258-5e44d218dba6 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:23.321616 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9c878e90-aa41-45f3-9258-5e44d218dba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:23.334528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c878e90-aa41-45f3-9258-5e44d218dba6 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:23.339984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:23.340679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:23.340962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:23.341233 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] PATH_INFO: `/v3/projects/edcdd3a879c343d2bf87c56837522adc` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:23.342518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edcdd3a879c343d2bf87c56837522adc)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:23.344512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:23.393397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edcdd3a879c343d2bf87c56837522adc', 'request_id': 'req-95ca55ba-8667-4417-a556-d0ab29350035'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:23.399584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edcdd3a879c343d2bf87c56837522adc', 'request_id': 'req-95ca55ba-8667-4417-a556-d0ab29350035'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:23.408842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edcdd3a879c343d2bf87c56837522adc', 'request_id': 'req-95ca55ba-8667-4417-a556-d0ab29350035'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:23.419802 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] The token cache is being invalidate because project edcdd3a879c343d2bf87c56837522adc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:10:23.420087 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:23.420764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:23.422290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1280/2555] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:10:23 2022] DELETE /identity/v3/projects/edcdd3a879c343d2bf87c56837522adc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:23.426796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732091de-b048-4b05-bf3b-65d5fffd661d service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:23.437191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732091de-b048-4b05-bf3b-65d5fffd661d service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:23.500395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:23.501073 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:23.501350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:23.501614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] PATH_INFO: `/v3/users/2151ef01cd144ccc8a80b81f0ba9efd8` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:23.502911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2151ef01cd144ccc8a80b81f0ba9efd8)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:23.504948 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:23.507575 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:23.560873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2151ef01cd144ccc8a80b81f0ba9efd8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:23.575536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2151ef01cd144ccc8a80b81f0ba9efd8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:23.586540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2151ef01cd144ccc8a80b81f0ba9efd8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:23.587269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2151ef01cd144ccc8a80b81f0ba9efd8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:23.587859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-0979877b-c613-446d-b3d2-b40a56c3f5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2151ef01cd144ccc8a80b81f0ba9efd8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:23.603068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1276/2556] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:10:23 2022] DELETE /identity/v3/users/2151ef01cd144ccc8a80b81f0ba9efd8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:23.917180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:23.919375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:23.935983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca55ba-8667-4417-a556-d0ab29350035 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:23.955118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:24.008592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.009158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.009413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.009715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] PATH_INFO: `/v3/projects/c31ad0b96be84abbb0f17cd81646b563` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.010955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31ad0b96be84abbb0f17cd81646b563)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:24.012923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:24.063200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31ad0b96be84abbb0f17cd81646b563', 'request_id': 'req-322f760c-38d0-4c0c-8daa-02af55a59752'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:24.069895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31ad0b96be84abbb0f17cd81646b563', 'request_id': 'req-322f760c-38d0-4c0c-8daa-02af55a59752'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:24.079567 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31ad0b96be84abbb0f17cd81646b563', 'request_id': 'req-322f760c-38d0-4c0c-8daa-02af55a59752'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:24.094736 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] The token cache is being invalidate because project c31ad0b96be84abbb0f17cd81646b563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:10:24.095374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:24.096322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-322f760c-38d0-4c0c-8daa-02af55a59752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:24.098245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1281/2557] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:10:23 2022] DELETE /identity/v3/projects/c31ad0b96be84abbb0f17cd81646b563 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:24.103755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba8b2-634b-4339-87c2-b05499df4ba0 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.104360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba8b2-634b-4339-87c2-b05499df4ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.104360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba8b2-634b-4339-87c2-b05499df4ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.126335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-f0fba8b2-634b-4339-87c2-b05499df4ba0 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:24.127586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f0fba8b2-634b-4339-87c2-b05499df4ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:24.190395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1277/2558] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:24.198671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77383a-3a8c-44ae-a845-b025f98868ac None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.198961 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77383a-3a8c-44ae-a845-b025f98868ac None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.199242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77383a-3a8c-44ae-a845-b025f98868ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.219605 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-4d77383a-3a8c-44ae-a845-b025f98868ac None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:24.220848 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-4d77383a-3a8c-44ae-a845-b025f98868ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:24.229431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1282/2559] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:24.233907 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba8b2-634b-4339-87c2-b05499df4ba0 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:24.244251 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba8b2-634b-4339-87c2-b05499df4ba0 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:24.249351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e58493-3572-496a-a8bc-3b58e3aa05a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.250038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e58493-3572-496a-a8bc-3b58e3aa05a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.250316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e58493-3572-496a-a8bc-3b58e3aa05a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.250587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e58493-3572-496a-a8bc-3b58e3aa05a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.251294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e58493-3572-496a-a8bc-3b58e3aa05a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:10:24.251595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e58493-3572-496a-a8bc-3b58e3aa05a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:24.253701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e58493-3572-496a-a8bc-3b58e3aa05a1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:24.266619 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1278/2560] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:10:24.271160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77383a-3a8c-44ae-a845-b025f98868ac None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:24.280730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77383a-3a8c-44ae-a845-b025f98868ac None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:24.285947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0312a-869d-497d-8032-139a80011e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.286573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0312a-869d-497d-8032-139a80011e1c admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.286852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0312a-869d-497d-8032-139a80011e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.287129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0312a-869d-497d-8032-139a80011e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.287796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0312a-869d-497d-8032-139a80011e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:24.289788 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0312a-869d-497d-8032-139a80011e1c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:24.319658 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1283/2561] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:10:24 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:24.324172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e58493-3572-496a-a8bc-3b58e3aa05a1 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:24.334673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e58493-3572-496a-a8bc-3b58e3aa05a1 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:24.394330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac9db7-6c05-47b7-8164-481161864dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.394960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac9db7-6c05-47b7-8164-481161864dad admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.395241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac9db7-6c05-47b7-8164-481161864dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.395512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac9db7-6c05-47b7-8164-481161864dad admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.396184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac9db7-6c05-47b7-8164-481161864dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:24.398167 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaac9db7-6c05-47b7-8164-481161864dad admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:24.415312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:24.422920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1279/2562] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:10:24 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:24.427641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0312a-869d-497d-8032-139a80011e1c admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:24.438535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0312a-869d-497d-8032-139a80011e1c admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:24.443487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f4598-1d85-4c80-b7bc-96783862cf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.444109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f4598-1d85-4c80-b7bc-96783862cf62 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.444414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f4598-1d85-4c80-b7bc-96783862cf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.444688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f4598-1d85-4c80-b7bc-96783862cf62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.445377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f4598-1d85-4c80-b7bc-96783862cf62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:24.447051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f4598-1d85-4c80-b7bc-96783862cf62 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:24.458216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1284/2563] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:24.462515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac9db7-6c05-47b7-8164-481161864dad admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:24.473112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac9db7-6c05-47b7-8164-481161864dad admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:24.478466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4708ecce-4ecd-48d4-aeb7-678b03124499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.479094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708ecce-4ecd-48d4-aeb7-678b03124499 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.479811 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708ecce-4ecd-48d4-aeb7-678b03124499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.480088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4708ecce-4ecd-48d4-aeb7-678b03124499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.480953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708ecce-4ecd-48d4-aeb7-678b03124499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:24.482447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4708ecce-4ecd-48d4-aeb7-678b03124499 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:24.493960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1280/2564] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:24.498355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f4598-1d85-4c80-b7bc-96783862cf62 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:24.510202 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f4598-1d85-4c80-b7bc-96783862cf62 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:24.515893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a34c2-d0df-4904-8a95-a0e1902fbd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.516531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a34c2-d0df-4904-8a95-a0e1902fbd8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.516816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a34c2-d0df-4904-8a95-a0e1902fbd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.517089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a34c2-d0df-4904-8a95-a0e1902fbd8f admin admin] PATH_INFO: `/v3/projects/230fcfe1173f4944aef4045f2f77c4b3/users/bde03e76d8764441a46dfa5070c50209/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.533038 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a34c2-d0df-4904-8a95-a0e1902fbd8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=230fcfe1173f4944aef4045f2f77c4b3, user_id=bde03e76d8764441a46dfa5070c50209, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:24.535835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a34c2-d0df-4904-8a95-a0e1902fbd8f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:24.553405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1285/2565] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:24 2022] PUT /identity/v3/projects/230fcfe1173f4944aef4045f2f77c4b3/users/bde03e76d8764441a46dfa5070c50209/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:24.567096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4ec63-d260-496a-a4ee-a075dd459a75 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.567419 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4ec63-d260-496a-a4ee-a075dd459a75 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.567905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4ec63-d260-496a-a4ee-a075dd459a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.609968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a8c4ec63-d260-496a-a4ee-a075dd459a75 None None] MFA Rules not processed for user `bde03e76d8764441a46dfa5070c50209`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:24.611756 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a8c4ec63-d260-496a-a4ee-a075dd459a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:24.685975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1281/2566] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:24.690508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a34c2-d0df-4904-8a95-a0e1902fbd8f admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:24.750279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:24.766692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a34c2-d0df-4904-8a95-a0e1902fbd8f admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:24.772043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3888006-d014-4fe6-ad77-d72fee973387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.772785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3888006-d014-4fe6-ad77-d72fee973387 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.773058 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3888006-d014-4fe6-ad77-d72fee973387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.773328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3888006-d014-4fe6-ad77-d72fee973387 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.773984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3888006-d014-4fe6-ad77-d72fee973387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:24.775946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3888006-d014-4fe6-ad77-d72fee973387 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:24.809536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1286/2567] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:10:24 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:24.810977 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4ec63-d260-496a-a4ee-a075dd459a75 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:24.822754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4ec63-d260-496a-a4ee-a075dd459a75 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:24.891670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bb801-f8b5-4305-af48-5c26692da591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.892346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bb801-f8b5-4305-af48-5c26692da591 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.892653 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bb801-f8b5-4305-af48-5c26692da591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.892921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bb801-f8b5-4305-af48-5c26692da591 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.893620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34bb801-f8b5-4305-af48-5c26692da591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:24.895590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34bb801-f8b5-4305-af48-5c26692da591 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:24.919717 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1282/2568] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:10:24 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:24.924265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3888006-d014-4fe6-ad77-d72fee973387 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:24.936846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3888006-d014-4fe6-ad77-d72fee973387 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:24.941997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae97107-9c85-493a-a068-c118efc22bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.942645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae97107-9c85-493a-a068-c118efc22bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.942925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae97107-9c85-493a-a068-c118efc22bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.943199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae97107-9c85-493a-a068-c118efc22bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.943917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae97107-9c85-493a-a068-c118efc22bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:24.945590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae97107-9c85-493a-a068-c118efc22bf6 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:24.957459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1287/2569] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:24.962050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bb801-f8b5-4305-af48-5c26692da591 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:24.973215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bb801-f8b5-4305-af48-5c26692da591 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:24.978937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50c42c-b623-449f-b79c-5d9811962922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:24.979909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50c42c-b623-449f-b79c-5d9811962922 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:24.980258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50c42c-b623-449f-b79c-5d9811962922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:24.980629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50c42c-b623-449f-b79c-5d9811962922 admin admin] PATH_INFO: `/v3/projects/27add77e6aa14530ba66328b40726177/users/7632078cfbb74e1682242ba2161a83ed/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:24.999799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50c42c-b623-449f-b79c-5d9811962922 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27add77e6aa14530ba66328b40726177, user_id=7632078cfbb74e1682242ba2161a83ed, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:25.002547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50c42c-b623-449f-b79c-5d9811962922 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:25.017784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1283/2570] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:24 2022] PUT /identity/v3/projects/27add77e6aa14530ba66328b40726177/users/7632078cfbb74e1682242ba2161a83ed/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:25.022466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae97107-9c85-493a-a068-c118efc22bf6 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:25.033817 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae97107-9c85-493a-a068-c118efc22bf6 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:25.093914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4ea39b-4c48-4f6e-aa36-c196581ca37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:25.094592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ea39b-4c48-4f6e-aa36-c196581ca37c admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:25.094871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ea39b-4c48-4f6e-aa36-c196581ca37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:25.095111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4ea39b-4c48-4f6e-aa36-c196581ca37c admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:25.095849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ea39b-4c48-4f6e-aa36-c196581ca37c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:25.097539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4ea39b-4c48-4f6e-aa36-c196581ca37c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:25.108564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1288/2571] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:25.112704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50c42c-b623-449f-b79c-5d9811962922 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:25.123755 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50c42c-b623-449f-b79c-5d9811962922 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:25.128710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d2690-bbf2-41e0-aa00-92947276563c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:25.129376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2690-bbf2-41e0-aa00-92947276563c admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:25.129654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2690-bbf2-41e0-aa00-92947276563c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:25.129926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2690-bbf2-41e0-aa00-92947276563c admin admin] PATH_INFO: `/v3/projects/27add77e6aa14530ba66328b40726177/users/7632078cfbb74e1682242ba2161a83ed/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:25.132035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d2690-bbf2-41e0-aa00-92947276563c admin admin] RBAC: Authorizing `identity:create_grant(project_id=27add77e6aa14530ba66328b40726177, user_id=7632078cfbb74e1682242ba2161a83ed, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:25.134723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d2690-bbf2-41e0-aa00-92947276563c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:25.145976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1284/2572] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:25 2022] PUT /identity/v3/projects/27add77e6aa14530ba66328b40726177/users/7632078cfbb74e1682242ba2161a83ed/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:25.157834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7944ec-ff77-4daf-b821-2237dbabfe37 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:25.158127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7944ec-ff77-4daf-b821-2237dbabfe37 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:25.158390 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7944ec-ff77-4daf-b821-2237dbabfe37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:25.189136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8f7944ec-ff77-4daf-b821-2237dbabfe37 None None] MFA Rules not processed for user `7632078cfbb74e1682242ba2161a83ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:25.190446 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8f7944ec-ff77-4daf-b821-2237dbabfe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:25.274071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1289/2573] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:25.286454 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d2690-bbf2-41e0-aa00-92947276563c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:25.288433 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-827d2690-bbf2-41e0-aa00-92947276563c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:25.323951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:25.348871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7944ec-ff77-4daf-b821-2237dbabfe37 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:25.350374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8f7944ec-ff77-4daf-b821-2237dbabfe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:25.355645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d2690-bbf2-41e0-aa00-92947276563c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:25.359561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5d2b3-d33a-4fa4-8a7b-2d8d06fa94ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:25.360189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5d2b3-d33a-4fa4-8a7b-2d8d06fa94ca service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:25.360637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5d2b3-d33a-4fa4-8a7b-2d8d06fa94ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:25.360822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5d2b3-d33a-4fa4-8a7b-2d8d06fa94ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:25.371321 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5d2b3-d33a-4fa4-8a7b-2d8d06fa94ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:25.372847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5d2b3-d33a-4fa4-8a7b-2d8d06fa94ca service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:25.383795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1285/2574] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:25.402641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:25.415984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7944ec-ff77-4daf-b821-2237dbabfe37 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:25.419681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904fbbb-03ba-4b2e-bdda-9ec2af9d2a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:25.420315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904fbbb-03ba-4b2e-bdda-9ec2af9d2a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:25.420612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904fbbb-03ba-4b2e-bdda-9ec2af9d2a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:25.420887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904fbbb-03ba-4b2e-bdda-9ec2af9d2a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:25.422816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f904fbbb-03ba-4b2e-bdda-9ec2af9d2a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:25.435858 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904fbbb-03ba-4b2e-bdda-9ec2af9d2a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:25.437352 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904fbbb-03ba-4b2e-bdda-9ec2af9d2a35 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:25.447663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1290/2575] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:25.474771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5d2b3-d33a-4fa4-8a7b-2d8d06fa94ca service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:25.485109 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5d2b3-d33a-4fa4-8a7b-2d8d06fa94ca service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:25.489483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafed724-ca55-4276-b207-d22d873796af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:25.490487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed724-ca55-4276-b207-d22d873796af service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:25.490757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed724-ca55-4276-b207-d22d873796af service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:25.491113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafed724-ca55-4276-b207-d22d873796af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:25.555304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafed724-ca55-4276-b207-d22d873796af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:25.556702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafed724-ca55-4276-b207-d22d873796af service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:25.566812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1286/2576] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:25.863389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904fbbb-03ba-4b2e-bdda-9ec2af9d2a35 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:25.933417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904fbbb-03ba-4b2e-bdda-9ec2af9d2a35 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:25.938173 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:25.938823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:25.939104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:25.939380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] PATH_INFO: `/v3/users/bde03e76d8764441a46dfa5070c50209` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:25.940716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bde03e76d8764441a46dfa5070c50209)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:25.942700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:25.994929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bde03e76d8764441a46dfa5070c50209'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:25.996152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bde03e76d8764441a46dfa5070c50209'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.005799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bde03e76d8764441a46dfa5070c50209'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.020892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bde03e76d8764441a46dfa5070c50209'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.028516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bde03e76d8764441a46dfa5070c50209'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.035471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1291/2577] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:10:25 2022] DELETE /identity/v3/users/bde03e76d8764441a46dfa5070c50209 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:26.040640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1814e4-5451-449b-bb6e-e32a3c72ca6b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:26.041022 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1814e4-5451-449b-bb6e-e32a3c72ca6b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:26.041415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1814e4-5451-449b-bb6e-e32a3c72ca6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:26.063706 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-bf1814e4-5451-449b-bb6e-e32a3c72ca6b None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:26.065169 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-bf1814e4-5451-449b-bb6e-e32a3c72ca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:26.134777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1287/2578] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:26.144705 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:26.146279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:26.208186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb291922-8bc9-452d-b84b-2a854acba713 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:26.212192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcaebe-33d4-4a9d-803d-3fb70f911bea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:26.212858 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcaebe-33d4-4a9d-803d-3fb70f911bea service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:26.213140 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcaebe-33d4-4a9d-803d-3fb70f911bea service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:26.213417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcaebe-33d4-4a9d-803d-3fb70f911bea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:26.223326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcaebe-33d4-4a9d-803d-3fb70f911bea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:26.224887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcaebe-33d4-4a9d-803d-3fb70f911bea service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:26.234931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1292/2579] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:26.526590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1814e4-5451-449b-bb6e-e32a3c72ca6b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:26.528047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-bf1814e4-5451-449b-bb6e-e32a3c72ca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:26.541875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1814e4-5451-449b-bb6e-e32a3c72ca6b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:26.547315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:26.548033 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:26.548508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:26.548787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] PATH_INFO: `/v3/projects/230fcfe1173f4944aef4045f2f77c4b3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:26.550004 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=230fcfe1173f4944aef4045f2f77c4b3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:26.551978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:26.605972 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '230fcfe1173f4944aef4045f2f77c4b3', 'request_id': 'req-eb9f57eb-c486-4aef-8895-5a00d16558f2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.613844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '230fcfe1173f4944aef4045f2f77c4b3', 'request_id': 'req-eb9f57eb-c486-4aef-8895-5a00d16558f2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.624093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '230fcfe1173f4944aef4045f2f77c4b3', 'request_id': 'req-eb9f57eb-c486-4aef-8895-5a00d16558f2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.628232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:26.636943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] The token cache is being invalidate because project 230fcfe1173f4944aef4045f2f77c4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:10:26.637227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.638209 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.641171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1288/2580] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:10:26 2022] DELETE /identity/v3/projects/230fcfe1173f4944aef4045f2f77c4b3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:26.645584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcaebe-33d4-4a9d-803d-3fb70f911bea service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:26.656371 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcaebe-33d4-4a9d-803d-3fb70f911bea service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:26.660588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:26.717613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:26.718257 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:26.718536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:26.718809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] PATH_INFO: `/v3/users/7632078cfbb74e1682242ba2161a83ed` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:26.720027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7632078cfbb74e1682242ba2161a83ed)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:26.722031 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:26.777141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7632078cfbb74e1682242ba2161a83ed'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.777815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7632078cfbb74e1682242ba2161a83ed'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.785954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7632078cfbb74e1682242ba2161a83ed'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.797852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7632078cfbb74e1682242ba2161a83ed'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.804920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-8e80f21e-9ea2-4f09-ae30-0a5770773e73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7632078cfbb74e1682242ba2161a83ed'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:26.811099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1293/2581] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:10:26 2022] DELETE /identity/v3/users/7632078cfbb74e1682242ba2161a83ed => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:27.103947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:27.105494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:27.119959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f57eb-c486-4aef-8895-5a00d16558f2 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:27.177968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:27.178639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.178791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.179070 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] PATH_INFO: `/v3/projects/27add77e6aa14530ba66328b40726177` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.180236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] RBAC: Authorizing `identity:delete_project(project_id=27add77e6aa14530ba66328b40726177)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:27.182183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:27.231572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27add77e6aa14530ba66328b40726177', 'request_id': 'req-520dcbd8-6083-4861-bc4c-33502b68edce'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:27.237877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27add77e6aa14530ba66328b40726177', 'request_id': 'req-520dcbd8-6083-4861-bc4c-33502b68edce'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:27.247880 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27add77e6aa14530ba66328b40726177', 'request_id': 'req-520dcbd8-6083-4861-bc4c-33502b68edce'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:27.258912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] The token cache is being invalidate because project 27add77e6aa14530ba66328b40726177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:10:27.259196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:27.259922 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-520dcbd8-6083-4861-bc4c-33502b68edce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:27.261473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1289/2582] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:10:27 2022] DELETE /identity/v3/projects/27add77e6aa14530ba66328b40726177 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:27.266750 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a983d-f23e-4188-b6bc-e96af2350f63 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.267269 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a983d-f23e-4188-b6bc-e96af2350f63 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.267387 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a983d-f23e-4188-b6bc-e96af2350f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.287839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-302a983d-f23e-4188-b6bc-e96af2350f63 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:27.289118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-302a983d-f23e-4188-b6bc-e96af2350f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:27.354170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1294/2583] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:27.363676 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a4c42-30ae-4dd3-9f59-559db0bc3837 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.364016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a4c42-30ae-4dd3-9f59-559db0bc3837 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.364199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a4c42-30ae-4dd3-9f59-559db0bc3837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.384943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1a8a4c42-30ae-4dd3-9f59-559db0bc3837 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:27.386162 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1a8a4c42-30ae-4dd3-9f59-559db0bc3837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:27.395158 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1290/2584] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:27.399742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a983d-f23e-4188-b6bc-e96af2350f63 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:27.409976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a983d-f23e-4188-b6bc-e96af2350f63 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:27.414777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b310e6-20ed-4d7c-8035-12ae9ed6bb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:27.415394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b310e6-20ed-4d7c-8035-12ae9ed6bb43 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.415666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b310e6-20ed-4d7c-8035-12ae9ed6bb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.415934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b310e6-20ed-4d7c-8035-12ae9ed6bb43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.416661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b310e6-20ed-4d7c-8035-12ae9ed6bb43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:10:27.416966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b310e6-20ed-4d7c-8035-12ae9ed6bb43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:27.418574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b310e6-20ed-4d7c-8035-12ae9ed6bb43 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:27.432496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1295/2585] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:10:27.437152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a4c42-30ae-4dd3-9f59-559db0bc3837 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:27.446861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a4c42-30ae-4dd3-9f59-559db0bc3837 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:27.451883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ce870-9036-42a2-aa2e-3dd07c029db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:27.452561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce870-9036-42a2-aa2e-3dd07c029db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.452835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce870-9036-42a2-aa2e-3dd07c029db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.453112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce870-9036-42a2-aa2e-3dd07c029db3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.453806 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ce870-9036-42a2-aa2e-3dd07c029db3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:27.455664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ce870-9036-42a2-aa2e-3dd07c029db3 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:27.490149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1291/2586] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:10:27 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:27.494748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b310e6-20ed-4d7c-8035-12ae9ed6bb43 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:27.505241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b310e6-20ed-4d7c-8035-12ae9ed6bb43 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:27.515289 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:27.571065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bce49a-9272-4118-8f9a-3cae9caf2e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:27.571726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bce49a-9272-4118-8f9a-3cae9caf2e21 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.572014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bce49a-9272-4118-8f9a-3cae9caf2e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.572266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bce49a-9272-4118-8f9a-3cae9caf2e21 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.573077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bce49a-9272-4118-8f9a-3cae9caf2e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:27.574979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bce49a-9272-4118-8f9a-3cae9caf2e21 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:27.597781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1296/2587] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:10:27 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:27.602086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ce870-9036-42a2-aa2e-3dd07c029db3 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:27.614537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ce870-9036-42a2-aa2e-3dd07c029db3 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:27.619870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390842d8-55e3-4540-829f-9a0f572614f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:27.620600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390842d8-55e3-4540-829f-9a0f572614f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.620875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390842d8-55e3-4540-829f-9a0f572614f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.621147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390842d8-55e3-4540-829f-9a0f572614f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.621906 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390842d8-55e3-4540-829f-9a0f572614f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:27.623562 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390842d8-55e3-4540-829f-9a0f572614f8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:27.634565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1292/2588] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:10:27.638687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bce49a-9272-4118-8f9a-3cae9caf2e21 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:27.649929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bce49a-9272-4118-8f9a-3cae9caf2e21 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:27.659184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8327f22-e1ab-4abc-8ec3-275297b03f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:27.659952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327f22-e1ab-4abc-8ec3-275297b03f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.660234 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327f22-e1ab-4abc-8ec3-275297b03f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.660531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8327f22-e1ab-4abc-8ec3-275297b03f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.661231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327f22-e1ab-4abc-8ec3-275297b03f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:27.662882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8327f22-e1ab-4abc-8ec3-275297b03f24 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:27.675896 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1297/2589] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:27.680886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390842d8-55e3-4540-829f-9a0f572614f8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:27.695034 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390842d8-55e3-4540-829f-9a0f572614f8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:27.701133 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350393d-642c-498c-84df-65b1762f78bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:27.701805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350393d-642c-498c-84df-65b1762f78bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.702088 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350393d-642c-498c-84df-65b1762f78bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.702364 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350393d-642c-498c-84df-65b1762f78bf admin admin] PATH_INFO: `/v3/projects/dc064c0d4c1d42a5b8184881af991cee/users/042bbeb01736437cb904ea18356f3072/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.721495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350393d-642c-498c-84df-65b1762f78bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc064c0d4c1d42a5b8184881af991cee, user_id=042bbeb01736437cb904ea18356f3072, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:27.724226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350393d-642c-498c-84df-65b1762f78bf admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:27.739244 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1293/2590] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:27 2022] PUT /identity/v3/projects/dc064c0d4c1d42a5b8184881af991cee/users/042bbeb01736437cb904ea18356f3072/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:10:27.751474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c28d8-2fc6-4f62-bc91-fd2f2f255a67 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.751760 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c28d8-2fc6-4f62-bc91-fd2f2f255a67 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.752028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c28d8-2fc6-4f62-bc91-fd2f2f255a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.783473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8f9c28d8-2fc6-4f62-bc91-fd2f2f255a67 None None] MFA Rules not processed for user `042bbeb01736437cb904ea18356f3072`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:27.785072 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8f9c28d8-2fc6-4f62-bc91-fd2f2f255a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:27.871877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1298/2591] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:27.885046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350393d-642c-498c-84df-65b1762f78bf admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:27.887662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7350393d-642c-498c-84df-65b1762f78bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:27.915517 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:27.965950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350393d-642c-498c-84df-65b1762f78bf admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:27.970118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82a075-030b-4d79-a056-56c04c841824 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:27.970758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82a075-030b-4d79-a056-56c04c841824 service glance] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:27.971044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82a075-030b-4d79-a056-56c04c841824 service glance] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:27.971318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82a075-030b-4d79-a056-56c04c841824 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:27.981630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82a075-030b-4d79-a056-56c04c841824 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:27.983271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82a075-030b-4d79-a056-56c04c841824 service glance] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:27.994812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1294/2592] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:28.722014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c28d8-2fc6-4f62-bc91-fd2f2f255a67 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:28.723932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8f9c28d8-2fc6-4f62-bc91-fd2f2f255a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:28.789152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c28d8-2fc6-4f62-bc91-fd2f2f255a67 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:28.793532 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd1155-c383-4086-836d-d91bfe204dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:28.794218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd1155-c383-4086-836d-d91bfe204dba service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:28.794510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd1155-c383-4086-836d-d91bfe204dba service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:28.794787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fd1155-c383-4086-836d-d91bfe204dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:28.797190 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-29fd1155-c383-4086-836d-d91bfe204dba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:28.808418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd1155-c383-4086-836d-d91bfe204dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:28.809893 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fd1155-c383-4086-836d-d91bfe204dba service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:28.819701 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1299/2593] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:29.100745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82a075-030b-4d79-a056-56c04c841824 service glance] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:29.102309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5f82a075-030b-4d79-a056-56c04c841824 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:29.182057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82a075-030b-4d79-a056-56c04c841824 service glance] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:29.187626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa023-a634-4a8d-9722-36ca7b8043a4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:29.188259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa023-a634-4a8d-9722-36ca7b8043a4 service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:29.188571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa023-a634-4a8d-9722-36ca7b8043a4 service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:29.188867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffa023-a634-4a8d-9722-36ca7b8043a4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:29.190850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-17ffa023-a634-4a8d-9722-36ca7b8043a4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:29.204094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffa023-a634-4a8d-9722-36ca7b8043a4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:29.206306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffa023-a634-4a8d-9722-36ca7b8043a4 service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:29.221616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1295/2594] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:10:36.547343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd1155-c383-4086-836d-d91bfe204dba service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:36.549265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-29fd1155-c383-4086-836d-d91bfe204dba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:36.585847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:36.644059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fd1155-c383-4086-836d-d91bfe204dba service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:36.649966 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98d387-9919-44c7-ab57-9b761d7918de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:36.650612 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98d387-9919-44c7-ab57-9b761d7918de service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:36.650898 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98d387-9919-44c7-ab57-9b761d7918de service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:36.651156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98d387-9919-44c7-ab57-9b761d7918de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:36.653096 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-fc98d387-9919-44c7-ab57-9b761d7918de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:36.720768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98d387-9919-44c7-ab57-9b761d7918de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:36.722621 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc98d387-9919-44c7-ab57-9b761d7918de service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:36.735374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1300/2595] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:39.213620 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=49 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:44.795606 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa023-a634-4a8d-9722-36ca7b8043a4 service placement] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:44.868786 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffa023-a634-4a8d-9722-36ca7b8043a4 service placement] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:44.873559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:44.874208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:44.874488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:44.874764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] PATH_INFO: `/v3/users/042bbeb01736437cb904ea18356f3072` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:44.875967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=042bbeb01736437cb904ea18356f3072)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:44.877923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:44.927508 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042bbeb01736437cb904ea18356f3072'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:44.939540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042bbeb01736437cb904ea18356f3072'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:44.946708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042bbeb01736437cb904ea18356f3072'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:44.947488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042bbeb01736437cb904ea18356f3072'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:44.948129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042bbeb01736437cb904ea18356f3072'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:44.963666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1296/2596] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:10:44 2022] DELETE /identity/v3/users/042bbeb01736437cb904ea18356f3072 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:44.968267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e599378-f117-43d3-a241-84439ce276c2 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:44.968932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e599378-f117-43d3-a241-84439ce276c2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:44.968932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e599378-f117-43d3-a241-84439ce276c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:44.990298 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-0e599378-f117-43d3-a241-84439ce276c2 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:44.991552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0e599378-f117-43d3-a241-84439ce276c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:45.058656 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1301/2597] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:45.069009 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:45.070521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:45.131750 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a176f-6016-4390-85ba-bb79d9e506ef admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:45.136985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133fb29-c8d9-4328-b0fc-d0c0cace43b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:45.137663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133fb29-c8d9-4328-b0fc-d0c0cace43b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:45.137914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133fb29-c8d9-4328-b0fc-d0c0cace43b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:45.138215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133fb29-c8d9-4328-b0fc-d0c0cace43b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:45.148854 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133fb29-c8d9-4328-b0fc-d0c0cace43b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:45.150315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133fb29-c8d9-4328-b0fc-d0c0cace43b9 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:45.161900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1297/2598] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:45.455185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e599378-f117-43d3-a241-84439ce276c2 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:45.456965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0e599378-f117-43d3-a241-84439ce276c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:45.475657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e599378-f117-43d3-a241-84439ce276c2 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:45.485903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:45.487081 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:45.487565 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:45.487990 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] PATH_INFO: `/v3/projects/dc064c0d4c1d42a5b8184881af991cee` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:45.489983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc064c0d4c1d42a5b8184881af991cee)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:45.493423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:45.551168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc064c0d4c1d42a5b8184881af991cee', 'request_id': 'req-527cf46b-0a5f-47d0-9700-d8dbb07deee5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:45.557828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc064c0d4c1d42a5b8184881af991cee', 'request_id': 'req-527cf46b-0a5f-47d0-9700-d8dbb07deee5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:45.561295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:45.571226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc064c0d4c1d42a5b8184881af991cee', 'request_id': 'req-527cf46b-0a5f-47d0-9700-d8dbb07deee5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:45.585851 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] The token cache is being invalidate because project dc064c0d4c1d42a5b8184881af991cee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:10:45.586167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:45.588536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-527cf46b-0a5f-47d0-9700-d8dbb07deee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:45.591649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1302/2599] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:10:45 2022] DELETE /identity/v3/projects/dc064c0d4c1d42a5b8184881af991cee => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:10:45.597651 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be6f76-62fd-4247-8810-689526e92a0e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:45.598096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be6f76-62fd-4247-8810-689526e92a0e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:45.598535 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be6f76-62fd-4247-8810-689526e92a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:45.624594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-47be6f76-62fd-4247-8810-689526e92a0e None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:45.625859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-47be6f76-62fd-4247-8810-689526e92a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:45.648931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:45.699290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1298/2600] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:45.707664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400a04f-f40d-4143-9d41-5dd55b1d1d87 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:45.707947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400a04f-f40d-4143-9d41-5dd55b1d1d87 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:45.708218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400a04f-f40d-4143-9d41-5dd55b1d1d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:45.732251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-f400a04f-f40d-4143-9d41-5dd55b1d1d87 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:45.733528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f400a04f-f40d-4143-9d41-5dd55b1d1d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:45.742903 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1303/2601] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:45.747635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be6f76-62fd-4247-8810-689526e92a0e None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:45.758450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be6f76-62fd-4247-8810-689526e92a0e None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:45.763583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947fda6-46ce-43a0-b4ca-b1b1b858c15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:45.764207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947fda6-46ce-43a0-b4ca-b1b1b858c15f admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:45.764512 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947fda6-46ce-43a0-b4ca-b1b1b858c15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:45.764787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947fda6-46ce-43a0-b4ca-b1b1b858c15f admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:45.765497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947fda6-46ce-43a0-b4ca-b1b1b858c15f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:10:45.765800 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947fda6-46ce-43a0-b4ca-b1b1b858c15f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:45.767414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7947fda6-46ce-43a0-b4ca-b1b1b858c15f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:45.780228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1299/2602] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:10:45.784907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400a04f-f40d-4143-9d41-5dd55b1d1d87 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:45.795122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400a04f-f40d-4143-9d41-5dd55b1d1d87 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:45.800569 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a3a4e-e912-4c27-819c-1ce37e19c34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:45.801204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3a4e-e912-4c27-819c-1ce37e19c34b admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:45.801509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3a4e-e912-4c27-819c-1ce37e19c34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:45.801752 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3a4e-e912-4c27-819c-1ce37e19c34b admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:45.802428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a3a4e-e912-4c27-819c-1ce37e19c34b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:45.804360 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a3a4e-e912-4c27-819c-1ce37e19c34b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:45.836743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1304/2603] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:10:45 2022] POST /identity/v3/projects => generated 357 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:45.842392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947fda6-46ce-43a0-b4ca-b1b1b858c15f admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:45.855637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947fda6-46ce-43a0-b4ca-b1b1b858c15f admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:45.921379 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f36e3-9c55-4949-94df-39a2aa2d5691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:45.922041 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f36e3-9c55-4949-94df-39a2aa2d5691 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:45.922315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f36e3-9c55-4949-94df-39a2aa2d5691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:45.922600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f36e3-9c55-4949-94df-39a2aa2d5691 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:45.923247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f36e3-9c55-4949-94df-39a2aa2d5691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:45.925204 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f36e3-9c55-4949-94df-39a2aa2d5691 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:45.952115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1300/2604] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:10:45 2022] POST /identity/v3/users => generated 279 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:45.956698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a3a4e-e912-4c27-819c-1ce37e19c34b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:45.968745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a3a4e-e912-4c27-819c-1ce37e19c34b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:45.973866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaa3dc-ae83-479b-8951-4faa2a1e239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:45.974525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaa3dc-ae83-479b-8951-4faa2a1e239f admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:45.974833 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaa3dc-ae83-479b-8951-4faa2a1e239f admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:45.975084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaa3dc-ae83-479b-8951-4faa2a1e239f admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:45.975805 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eaa3dc-ae83-479b-8951-4faa2a1e239f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:45.977566 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eaa3dc-ae83-479b-8951-4faa2a1e239f admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:45.991417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1305/2605] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:45.995780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f36e3-9c55-4949-94df-39a2aa2d5691 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:46.006883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f36e3-9c55-4949-94df-39a2aa2d5691 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:46.012806 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed9cfb-edc3-4a1a-be07-5854a77b266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:46.013441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed9cfb-edc3-4a1a-be07-5854a77b266c admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:46.013715 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed9cfb-edc3-4a1a-be07-5854a77b266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:46.013990 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed9cfb-edc3-4a1a-be07-5854a77b266c admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:46.014690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed9cfb-edc3-4a1a-be07-5854a77b266c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:46.016282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed9cfb-edc3-4a1a-be07-5854a77b266c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:46.028533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1301/2606] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:46.033239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaa3dc-ae83-479b-8951-4faa2a1e239f admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:46.044716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaa3dc-ae83-479b-8951-4faa2a1e239f admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:46.050527 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e48748-399f-4bfa-86a0-a5c1472a8fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:46.051227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e48748-399f-4bfa-86a0-a5c1472a8fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:46.051510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e48748-399f-4bfa-86a0-a5c1472a8fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:46.051781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e48748-399f-4bfa-86a0-a5c1472a8fdb admin admin] PATH_INFO: `/v3/projects/ed72365be8334455a49f9418d2f8f134/users/85801497c80e4ed1872802bfac5e24b5/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:46.068962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e48748-399f-4bfa-86a0-a5c1472a8fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed72365be8334455a49f9418d2f8f134, user_id=85801497c80e4ed1872802bfac5e24b5, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:46.071741 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e48748-399f-4bfa-86a0-a5c1472a8fdb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:46.088225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1306/2607] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:46 2022] PUT /identity/v3/projects/ed72365be8334455a49f9418d2f8f134/users/85801497c80e4ed1872802bfac5e24b5/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:46.099293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ce7ba-3df7-440c-8801-a9dfd6330d49 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:46.099573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ce7ba-3df7-440c-8801-a9dfd6330d49 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:46.099849 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ce7ba-3df7-440c-8801-a9dfd6330d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:46.135346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-cb5ce7ba-3df7-440c-8801-a9dfd6330d49 None None] MFA Rules not processed for user `85801497c80e4ed1872802bfac5e24b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:46.136616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-cb5ce7ba-3df7-440c-8801-a9dfd6330d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:46.221912 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1302/2608] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:46 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:46.270425 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e48748-399f-4bfa-86a0-a5c1472a8fdb admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:46.272517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e6e48748-399f-4bfa-86a0-a5c1472a8fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:46.346445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e48748-399f-4bfa-86a0-a5c1472a8fdb admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:46.350341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef2a27-bffd-429c-b4cf-c3b5644d40c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:46.350996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef2a27-bffd-429c-b4cf-c3b5644d40c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:46.351267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef2a27-bffd-429c-b4cf-c3b5644d40c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:46.351541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef2a27-bffd-429c-b4cf-c3b5644d40c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:46.372254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef2a27-bffd-429c-b4cf-c3b5644d40c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:46.375160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef2a27-bffd-429c-b4cf-c3b5644d40c0 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:46.393291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1307/2609] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:46 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:10:46.410085 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ce7ba-3df7-440c-8801-a9dfd6330d49 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:46.435473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:46.496087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ce7ba-3df7-440c-8801-a9dfd6330d49 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:46.501439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:46.502116 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:46.502428 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:46.502817 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] PATH_INFO: `/v3/users/85801497c80e4ed1872802bfac5e24b5` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:46.504159 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=85801497c80e4ed1872802bfac5e24b5)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:46.506587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:46.568570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85801497c80e4ed1872802bfac5e24b5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:46.584271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85801497c80e4ed1872802bfac5e24b5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:46.593678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85801497c80e4ed1872802bfac5e24b5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:46.594524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85801497c80e4ed1872802bfac5e24b5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:46.595237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85801497c80e4ed1872802bfac5e24b5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:46.615191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1303/2610] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:10:46 2022] DELETE /identity/v3/users/85801497c80e4ed1872802bfac5e24b5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:46.619932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5f600-7366-493a-a1bd-f16279766304 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:46.620184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5f600-7366-493a-a1bd-f16279766304 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:46.620505 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a5f600-7366-493a-a1bd-f16279766304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:46.643462 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-14a5f600-7366-493a-a1bd-f16279766304 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:46.645151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-14a5f600-7366-493a-a1bd-f16279766304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:46.730774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1308/2611] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:46.742136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:46.743954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:46.803881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e63ab-c4ef-4204-8389-3a2deb102fda admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:46.807745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189dfe9-0f53-4a5d-af7d-0e0d616457d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:46.808404 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189dfe9-0f53-4a5d-af7d-0e0d616457d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:46.808681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189dfe9-0f53-4a5d-af7d-0e0d616457d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:46.808948 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189dfe9-0f53-4a5d-af7d-0e0d616457d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:46.819400 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189dfe9-0f53-4a5d-af7d-0e0d616457d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:46.820982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189dfe9-0f53-4a5d-af7d-0e0d616457d4 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:46.831937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1304/2612] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:47.133375 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5f600-7366-493a-a1bd-f16279766304 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:47.135275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-14a5f600-7366-493a-a1bd-f16279766304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:47.148178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a5f600-7366-493a-a1bd-f16279766304 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:47.154443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:47.155240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.155734 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.156320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] PATH_INFO: `/v3/projects/ed72365be8334455a49f9418d2f8f134` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.158076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed72365be8334455a49f9418d2f8f134)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:47.161357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:47.190584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:47.221331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed72365be8334455a49f9418d2f8f134', 'request_id': 'req-095aa146-3d22-417a-870f-60dd6f7cfff4'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:47.229125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed72365be8334455a49f9418d2f8f134', 'request_id': 'req-095aa146-3d22-417a-870f-60dd6f7cfff4'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:47.239106 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed72365be8334455a49f9418d2f8f134', 'request_id': 'req-095aa146-3d22-417a-870f-60dd6f7cfff4'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:47.253018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] The token cache is being invalidate because project ed72365be8334455a49f9418d2f8f134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:10:47.253307 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:47.254089 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-095aa146-3d22-417a-870f-60dd6f7cfff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:10:47.256273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1309/2613] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:10:47 2022] DELETE /identity/v3/projects/ed72365be8334455a49f9418d2f8f134 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:10:47.262499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3b19-daad-4ffd-b7ae-eb0420f74ef9 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.263108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3b19-daad-4ffd-b7ae-eb0420f74ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.263108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d3b19-daad-4ffd-b7ae-eb0420f74ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.283429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-f84d3b19-daad-4ffd-b7ae-eb0420f74ef9 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:47.284688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-f84d3b19-daad-4ffd-b7ae-eb0420f74ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:47.319794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:47.349852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1305/2614] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:47.359265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed910f-dc3a-4b99-983c-88bd8745726b None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.359508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed910f-dc3a-4b99-983c-88bd8745726b None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.359841 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed910f-dc3a-4b99-983c-88bd8745726b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.384998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-b6ed910f-dc3a-4b99-983c-88bd8745726b None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:47.386266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b6ed910f-dc3a-4b99-983c-88bd8745726b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:47.398077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1310/2615] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:47.403047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3b19-daad-4ffd-b7ae-eb0420f74ef9 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:47.413208 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84d3b19-daad-4ffd-b7ae-eb0420f74ef9 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:47.418210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1434d0-4a94-4c57-b335-47cc25d50109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:47.418866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1434d0-4a94-4c57-b335-47cc25d50109 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.419142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1434d0-4a94-4c57-b335-47cc25d50109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.419417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1434d0-4a94-4c57-b335-47cc25d50109 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.420163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1434d0-4a94-4c57-b335-47cc25d50109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:10:47.420507 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1434d0-4a94-4c57-b335-47cc25d50109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:47.422140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1434d0-4a94-4c57-b335-47cc25d50109 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:47.435742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1306/2616] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:10:47.440832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed910f-dc3a-4b99-983c-88bd8745726b None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:47.453227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed910f-dc3a-4b99-983c-88bd8745726b None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:47.463870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b08d91-8b55-4930-8926-590004b43667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:47.464564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08d91-8b55-4930-8926-590004b43667 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.464853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08d91-8b55-4930-8926-590004b43667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.465125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08d91-8b55-4930-8926-590004b43667 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.465832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b08d91-8b55-4930-8926-590004b43667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:47.467820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b08d91-8b55-4930-8926-590004b43667 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:47.504580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1311/2617] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:10:47 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:10:47.507064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1434d0-4a94-4c57-b335-47cc25d50109 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:47.520970 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1434d0-4a94-4c57-b335-47cc25d50109 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:47.582378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2ecb8-f5c6-4788-bfe2-b7b81ca73a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:47.583017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2ecb8-f5c6-4788-bfe2-b7b81ca73a3f admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.583300 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2ecb8-f5c6-4788-bfe2-b7b81ca73a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.583571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2ecb8-f5c6-4788-bfe2-b7b81ca73a3f admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.584254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2ecb8-f5c6-4788-bfe2-b7b81ca73a3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:47.586283 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e2ecb8-f5c6-4788-bfe2-b7b81ca73a3f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:47.609083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1307/2618] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:10:47 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:10:47.613281 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b08d91-8b55-4930-8926-590004b43667 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:47.624527 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b08d91-8b55-4930-8926-590004b43667 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:47.629327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c3fc4-d907-4461-907e-17e4d7bea87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:47.629960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c3fc4-d907-4461-907e-17e4d7bea87d admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.630242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c3fc4-d907-4461-907e-17e4d7bea87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.630523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c3fc4-d907-4461-907e-17e4d7bea87d admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.631270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c3fc4-d907-4461-907e-17e4d7bea87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:47.632942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c3fc4-d907-4461-907e-17e4d7bea87d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:47.645052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1312/2619] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:47.649435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2ecb8-f5c6-4788-bfe2-b7b81ca73a3f admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:47.660397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2ecb8-f5c6-4788-bfe2-b7b81ca73a3f admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:47.665106 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e69f2-bd6e-4aae-afdc-4e3ecf5dec60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:47.665742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e69f2-bd6e-4aae-afdc-4e3ecf5dec60 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.666021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e69f2-bd6e-4aae-afdc-4e3ecf5dec60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.666328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e69f2-bd6e-4aae-afdc-4e3ecf5dec60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.667032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e69f2-bd6e-4aae-afdc-4e3ecf5dec60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:47.668685 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e69f2-bd6e-4aae-afdc-4e3ecf5dec60 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:47.680260 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1308/2620] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:10:47.685064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c3fc4-d907-4461-907e-17e4d7bea87d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:47.698195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c3fc4-d907-4461-907e-17e4d7bea87d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:47.703029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3e053-3412-4e88-94d9-7ac1cd900904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:47.703714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3e053-3412-4e88-94d9-7ac1cd900904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.703996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3e053-3412-4e88-94d9-7ac1cd900904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.704270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3e053-3412-4e88-94d9-7ac1cd900904 admin admin] PATH_INFO: `/v3/projects/48b4b97b8e30431189440e4dcb27a1c4/users/4f142903e9424690aafeb20cc166c43b/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.720392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3e053-3412-4e88-94d9-7ac1cd900904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b4b97b8e30431189440e4dcb27a1c4, user_id=4f142903e9424690aafeb20cc166c43b, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:47.723090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3e053-3412-4e88-94d9-7ac1cd900904 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:47.739053 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1313/2621] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:10:47 2022] PUT /identity/v3/projects/48b4b97b8e30431189440e4dcb27a1c4/users/4f142903e9424690aafeb20cc166c43b/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:10:47.750414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10181841-7ec1-4c53-977d-b58dbcae6f59 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.750692 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10181841-7ec1-4c53-977d-b58dbcae6f59 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.750960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10181841-7ec1-4c53-977d-b58dbcae6f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.782911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-10181841-7ec1-4c53-977d-b58dbcae6f59 None None] MFA Rules not processed for user `4f142903e9424690aafeb20cc166c43b`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:10:47.784637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-10181841-7ec1-4c53-977d-b58dbcae6f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:47.859316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1309/2622] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:10:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:10:47.872712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3e053-3412-4e88-94d9-7ac1cd900904 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:47.874723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7fc3e053-3412-4e88-94d9-7ac1cd900904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:47.950842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3e053-3412-4e88-94d9-7ac1cd900904 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:47.954618 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1ce24-e96d-4d7b-bffb-24d216a78fd5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:47.955325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1ce24-e96d-4d7b-bffb-24d216a78fd5 service glance] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:47.955573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1ce24-e96d-4d7b-bffb-24d216a78fd5 service glance] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:47.955927 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1ce24-e96d-4d7b-bffb-24d216a78fd5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:47.968184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1ce24-e96d-4d7b-bffb-24d216a78fd5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:47.969731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc1ce24-e96d-4d7b-bffb-24d216a78fd5 service glance] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:47.979987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1314/2623] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:10:57.855440 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:57.975279 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=69,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:10:58.375960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10181841-7ec1-4c53-977d-b58dbcae6f59 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:10:58.377439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-10181841-7ec1-4c53-977d-b58dbcae6f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:58.536530 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10181841-7ec1-4c53-977d-b58dbcae6f59 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:10:58.542119 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb532cd-cf1b-4c2c-9503-86bc706a2b2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:10:58.542748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb532cd-cf1b-4c2c-9503-86bc706a2b2b service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:10:58.543031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb532cd-cf1b-4c2c-9503-86bc706a2b2b service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:10:58.543301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb532cd-cf1b-4c2c-9503-86bc706a2b2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:10:58.545134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-bbb532cd-cf1b-4c2c-9503-86bc706a2b2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:10:58.559762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb532cd-cf1b-4c2c-9503-86bc706a2b2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:10:58.562150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb532cd-cf1b-4c2c-9503-86bc706a2b2b service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:10:58.574848 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1310/2624] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:10:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:04.208617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1ce24-e96d-4d7b-bffb-24d216a78fd5 service glance] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:04.210125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7bc1ce24-e96d-4d7b-bffb-24d216a78fd5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:04.317654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc1ce24-e96d-4d7b-bffb-24d216a78fd5 service glance] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:04.322044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc42c0-2972-4ac1-ab25-5cdfb0b87632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:04.322678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc42c0-2972-4ac1-ab25-5cdfb0b87632 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:04.322960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc42c0-2972-4ac1-ab25-5cdfb0b87632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:04.323227 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc42c0-2972-4ac1-ab25-5cdfb0b87632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:04.325385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-aebc42c0-2972-4ac1-ab25-5cdfb0b87632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:04.414971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc42c0-2972-4ac1-ab25-5cdfb0b87632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:04.416331 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc42c0-2972-4ac1-ab25-5cdfb0b87632 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:04.437074 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1315/2625] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:06.753021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7394f48-5e0e-47e7-a501-1f03624a78fa None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:06.753763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7394f48-5e0e-47e7-a501-1f03624a78fa None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:06.753763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7394f48-5e0e-47e7-a501-1f03624a78fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:06.804795 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a7394f48-5e0e-47e7-a501-1f03624a78fa None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:06.806125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a7394f48-5e0e-47e7-a501-1f03624a78fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:06.888622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1311/2626] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:11:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:06.948152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc42c0-2972-4ac1-ab25-5cdfb0b87632 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:06.949995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-aebc42c0-2972-4ac1-ab25-5cdfb0b87632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:07.013747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:07.037925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc42c0-2972-4ac1-ab25-5cdfb0b87632 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:07.044542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8f93a-c222-44ba-a196-e683802c82ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:07.045228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8f93a-c222-44ba-a196-e683802c82ec service swift] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:07.045587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8f93a-c222-44ba-a196-e683802c82ec service swift] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:07.045933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8f93a-c222-44ba-a196-e683802c82ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:07.055852 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8f93a-c222-44ba-a196-e683802c82ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:07.057647 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8f93a-c222-44ba-a196-e683802c82ec service swift] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:07.069339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1316/2627] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:11:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:11:08.342250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7394f48-5e0e-47e7-a501-1f03624a78fa None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:08.343769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a7394f48-5e0e-47e7-a501-1f03624a78fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:08.361839 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7394f48-5e0e-47e7-a501-1f03624a78fa None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:08.368042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:08.368824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:08.369166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:08.369551 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] PATH_INFO: `/v3/users/443696f19aec46fc9cd59bd9d61587f5` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:08.371258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=443696f19aec46fc9cd59bd9d61587f5)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:08.373458 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:08.427700 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:08.462776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '443696f19aec46fc9cd59bd9d61587f5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:08.482183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '443696f19aec46fc9cd59bd9d61587f5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:08.490067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '443696f19aec46fc9cd59bd9d61587f5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:08.490911 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '443696f19aec46fc9cd59bd9d61587f5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:08.491666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-6577dee9-1fd4-4040-8a9d-f1f33ed38ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '443696f19aec46fc9cd59bd9d61587f5'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:08.514791 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1312/2628] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:11:08 2022] DELETE /identity/v3/users/443696f19aec46fc9cd59bd9d61587f5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:08.786139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8f93a-c222-44ba-a196-e683802c82ec service swift] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:08.798590 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8f93a-c222-44ba-a196-e683802c82ec service swift] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:08.886857 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:08.887977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:08.888498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:08.888845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] PATH_INFO: `/v3/projects/5fd75f5d11444549bfa2d273599bb098` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:08.890323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd75f5d11444549bfa2d273599bb098)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:08.892551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:08.946011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd75f5d11444549bfa2d273599bb098', 'request_id': 'req-23d65eda-4ece-45a8-8d8b-7b1a00639694'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:08.952564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd75f5d11444549bfa2d273599bb098', 'request_id': 'req-23d65eda-4ece-45a8-8d8b-7b1a00639694'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:08.965139 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd75f5d11444549bfa2d273599bb098', 'request_id': 'req-23d65eda-4ece-45a8-8d8b-7b1a00639694'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:08.979758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] The token cache is being invalidate because project 5fd75f5d11444549bfa2d273599bb098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:11:08.980463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:08.981066 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-23d65eda-4ece-45a8-8d8b-7b1a00639694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:08.982888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1317/2629] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:11:08 2022] DELETE /identity/v3/projects/5fd75f5d11444549bfa2d273599bb098 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:08.992007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e2fa4-632b-461b-a927-9f9555937353 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:08.992913 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e2fa4-632b-461b-a927-9f9555937353 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:08.993266 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e2fa4-632b-461b-a927-9f9555937353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:09.060952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d22e2fa4-632b-461b-a927-9f9555937353 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:09.062307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d22e2fa4-632b-461b-a927-9f9555937353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:09.140524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1313/2630] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:09.149071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572beec3-7eaa-438c-8cba-6fd545b235c4 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:09.149518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572beec3-7eaa-438c-8cba-6fd545b235c4 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:09.149816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572beec3-7eaa-438c-8cba-6fd545b235c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:09.173207 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-572beec3-7eaa-438c-8cba-6fd545b235c4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:09.174487 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-572beec3-7eaa-438c-8cba-6fd545b235c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:09.187480 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1318/2631] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:09.190546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e2fa4-632b-461b-a927-9f9555937353 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:09.203125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e2fa4-632b-461b-a927-9f9555937353 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:09.208176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa652ebe-9da7-4db5-8540-1145baebc540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:09.208868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa652ebe-9da7-4db5-8540-1145baebc540 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:09.209126 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa652ebe-9da7-4db5-8540-1145baebc540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:09.209402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa652ebe-9da7-4db5-8540-1145baebc540 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:09.210125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa652ebe-9da7-4db5-8540-1145baebc540 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:11:09.210504 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa652ebe-9da7-4db5-8540-1145baebc540 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:09.212140 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa652ebe-9da7-4db5-8540-1145baebc540 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:09.226273 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1314/2632] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:11:09.231149 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572beec3-7eaa-438c-8cba-6fd545b235c4 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:09.242509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572beec3-7eaa-438c-8cba-6fd545b235c4 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:09.247370 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebbc3a-9647-48d4-a5fe-b343f29d279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:09.247997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebbc3a-9647-48d4-a5fe-b343f29d279a admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:09.248463 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebbc3a-9647-48d4-a5fe-b343f29d279a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:09.248738 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aebbc3a-9647-48d4-a5fe-b343f29d279a admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:09.249573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebbc3a-9647-48d4-a5fe-b343f29d279a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:09.251509 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aebbc3a-9647-48d4-a5fe-b343f29d279a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:09.305333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1319/2633] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:11:09 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:09.306638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa652ebe-9da7-4db5-8540-1145baebc540 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:09.320172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa652ebe-9da7-4db5-8540-1145baebc540 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:09.388824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee5e01-1c03-41c7-916c-f0b54bb18fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:09.389502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee5e01-1c03-41c7-916c-f0b54bb18fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:09.389778 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee5e01-1c03-41c7-916c-f0b54bb18fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:09.389998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee5e01-1c03-41c7-916c-f0b54bb18fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:09.390726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee5e01-1c03-41c7-916c-f0b54bb18fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:09.392739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee5e01-1c03-41c7-916c-f0b54bb18fa9 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:09.405245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:09.422141 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1315/2634] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:11:09 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:09.428339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebbc3a-9647-48d4-a5fe-b343f29d279a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:09.442640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aebbc3a-9647-48d4-a5fe-b343f29d279a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:09.452411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c123d3-6c0b-4cca-b5c0-75d2843695ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:09.453161 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c123d3-6c0b-4cca-b5c0-75d2843695ae admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:09.453513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c123d3-6c0b-4cca-b5c0-75d2843695ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:09.453878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c123d3-6c0b-4cca-b5c0-75d2843695ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:09.454663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c123d3-6c0b-4cca-b5c0-75d2843695ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:09.460723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c123d3-6c0b-4cca-b5c0-75d2843695ae admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:09.476137 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1320/2635] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:09.477668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee5e01-1c03-41c7-916c-f0b54bb18fa9 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:09.492256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee5e01-1c03-41c7-916c-f0b54bb18fa9 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:09.497126 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4158c2-5f3c-4b87-9b58-24942516a174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:09.497874 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4158c2-5f3c-4b87-9b58-24942516a174 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:09.498305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4158c2-5f3c-4b87-9b58-24942516a174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:09.498699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4158c2-5f3c-4b87-9b58-24942516a174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:09.499505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4158c2-5f3c-4b87-9b58-24942516a174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:09.501460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4158c2-5f3c-4b87-9b58-24942516a174 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:09.514867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1316/2636] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:09.519143 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c123d3-6c0b-4cca-b5c0-75d2843695ae admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:09.531859 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c123d3-6c0b-4cca-b5c0-75d2843695ae admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:09.540838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c871aa-1ab2-4a87-8dd3-4bc2537cbbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:09.542119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c871aa-1ab2-4a87-8dd3-4bc2537cbbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:09.542119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c871aa-1ab2-4a87-8dd3-4bc2537cbbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:09.542396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c871aa-1ab2-4a87-8dd3-4bc2537cbbe2 admin admin] PATH_INFO: `/v3/projects/353883808ec94809931fe6cbf22df097/users/bd8222285bb64463b92881ccb1ee49c0/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:09.566011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c871aa-1ab2-4a87-8dd3-4bc2537cbbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=353883808ec94809931fe6cbf22df097, user_id=bd8222285bb64463b92881ccb1ee49c0, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:09.568898 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c871aa-1ab2-4a87-8dd3-4bc2537cbbe2 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:09.579542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:09.593469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1321/2637] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:09 2022] PUT /identity/v3/projects/353883808ec94809931fe6cbf22df097/users/bd8222285bb64463b92881ccb1ee49c0/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:09.605672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d963242-ccce-44a9-bc40-a5da43ac10c1 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:09.606264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d963242-ccce-44a9-bc40-a5da43ac10c1 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:09.606554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d963242-ccce-44a9-bc40-a5da43ac10c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:09.665459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-1d963242-ccce-44a9-bc40-a5da43ac10c1 None None] MFA Rules not processed for user `bd8222285bb64463b92881ccb1ee49c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:09.667043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1d963242-ccce-44a9-bc40-a5da43ac10c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:09.775521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1317/2638] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:09.794183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c871aa-1ab2-4a87-8dd3-4bc2537cbbe2 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:09.796523 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-83c871aa-1ab2-4a87-8dd3-4bc2537cbbe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:09.930744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c871aa-1ab2-4a87-8dd3-4bc2537cbbe2 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:09.947560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44985d1-c228-42eb-aa6d-581d97d9f7b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:09.948226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44985d1-c228-42eb-aa6d-581d97d9f7b1 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:09.948529 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44985d1-c228-42eb-aa6d-581d97d9f7b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:09.948807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44985d1-c228-42eb-aa6d-581d97d9f7b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:09.962259 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44985d1-c228-42eb-aa6d-581d97d9f7b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:09.965188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44985d1-c228-42eb-aa6d-581d97d9f7b1 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:09.976825 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1322/2639] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:10.753608 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d963242-ccce-44a9-bc40-a5da43ac10c1 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:10.771087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d963242-ccce-44a9-bc40-a5da43ac10c1 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:10.777427 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c63da6-6491-4bb4-a293-542ed788d36a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:10.778290 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63da6-6491-4bb4-a293-542ed788d36a service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:10.778567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63da6-6491-4bb4-a293-542ed788d36a service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:10.779048 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63da6-6491-4bb4-a293-542ed788d36a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:10.780893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-44c63da6-6491-4bb4-a293-542ed788d36a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:10.796875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c63da6-6491-4bb4-a293-542ed788d36a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:10.799504 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c63da6-6491-4bb4-a293-542ed788d36a service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:10.812125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1318/2640] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:11:16.552946 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44985d1-c228-42eb-aa6d-581d97d9f7b1 service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:16.558732 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f44985d1-c228-42eb-aa6d-581d97d9f7b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:16.624895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44985d1-c228-42eb-aa6d-581d97d9f7b1 service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:16.630400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:16.631057 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:16.631367 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:16.631671 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] PATH_INFO: `/v3/users/4f142903e9424690aafeb20cc166c43b` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:16.632914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f142903e9424690aafeb20cc166c43b)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:16.634884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:16.685102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f142903e9424690aafeb20cc166c43b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:16.685753 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f142903e9424690aafeb20cc166c43b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:16.693350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f142903e9424690aafeb20cc166c43b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:16.706230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f142903e9424690aafeb20cc166c43b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:16.714072 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f142903e9424690aafeb20cc166c43b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:16.720208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1323/2641] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:11:16 2022] DELETE /identity/v3/users/4f142903e9424690aafeb20cc166c43b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:11:16.724998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45cbd6-2ee4-4ee5-bbba-e1e93e170177 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:16.725476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45cbd6-2ee4-4ee5-bbba-e1e93e170177 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:16.725599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45cbd6-2ee4-4ee5-bbba-e1e93e170177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:16.746828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-7d45cbd6-2ee4-4ee5-bbba-e1e93e170177 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:16.748450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7d45cbd6-2ee4-4ee5-bbba-e1e93e170177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:16.821156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1319/2642] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:11:16.831327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:16.832844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:16.838530 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:16.904008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c02234-3849-4d40-932c-8d817452e64b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:16.908377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51932f28-4cc7-47be-97b8-99b8dc168edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:16.909164 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932f28-4cc7-47be-97b8-99b8dc168edd service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:16.909455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932f28-4cc7-47be-97b8-99b8dc168edd service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:16.909732 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932f28-4cc7-47be-97b8-99b8dc168edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:16.919388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51932f28-4cc7-47be-97b8-99b8dc168edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:16.921014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51932f28-4cc7-47be-97b8-99b8dc168edd service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:16.932880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1324/2643] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:17.232374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45cbd6-2ee4-4ee5-bbba-e1e93e170177 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:17.234180 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7d45cbd6-2ee4-4ee5-bbba-e1e93e170177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:17.248360 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45cbd6-2ee4-4ee5-bbba-e1e93e170177 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:17.255303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:17.256096 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:17.256415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:17.256727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] PATH_INFO: `/v3/projects/48b4b97b8e30431189440e4dcb27a1c4` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:17.258089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48b4b97b8e30431189440e4dcb27a1c4)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:17.260052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:17.313196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b4b97b8e30431189440e4dcb27a1c4', 'request_id': 'req-e838fc4e-0318-4d83-b6ac-56f857547cd2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:17.315446 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:INSERT=2,keystone:SELECT=94,keystone:DELETE=4 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:17.321722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b4b97b8e30431189440e4dcb27a1c4', 'request_id': 'req-e838fc4e-0318-4d83-b6ac-56f857547cd2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:17.332877 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b4b97b8e30431189440e4dcb27a1c4', 'request_id': 'req-e838fc4e-0318-4d83-b6ac-56f857547cd2'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:17.345962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] The token cache is being invalidate because project 48b4b97b8e30431189440e4dcb27a1c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:11:17.346278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:17.346954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-e838fc4e-0318-4d83-b6ac-56f857547cd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:17.348728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1320/2644] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:11:17 2022] DELETE /identity/v3/projects/48b4b97b8e30431189440e4dcb27a1c4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:17.605945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51932f28-4cc7-47be-97b8-99b8dc168edd service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:17.607718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-51932f28-4cc7-47be-97b8-99b8dc168edd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:17.625353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51932f28-4cc7-47be-97b8-99b8dc168edd service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:17.692516 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:17.693165 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:17.693439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:17.693712 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] PATH_INFO: `/v3/users/bd8222285bb64463b92881ccb1ee49c0` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:17.694929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8222285bb64463b92881ccb1ee49c0)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:17.696950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:17.744220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8222285bb64463b92881ccb1ee49c0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:17.744933 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8222285bb64463b92881ccb1ee49c0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:17.753591 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8222285bb64463b92881ccb1ee49c0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:17.767071 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8222285bb64463b92881ccb1ee49c0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:17.774152 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8222285bb64463b92881ccb1ee49c0'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:17.780474 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1325/2645] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:11:17 2022] DELETE /identity/v3/users/bd8222285bb64463b92881ccb1ee49c0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:17.784971 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be072c-e920-4c33-bc8b-4fe6106c0660 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:17.785477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be072c-e920-4c33-bc8b-4fe6106c0660 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:17.785577 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97be072c-e920-4c33-bc8b-4fe6106c0660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:17.805682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-97be072c-e920-4c33-bc8b-4fe6106c0660 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:17.806916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-97be072c-e920-4c33-bc8b-4fe6106c0660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:17.871430 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1321/2646] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:17.881517 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:17.883103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:17.913092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:17.946551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f822d1c-4ad9-4c2e-b88a-057fa8cc2884 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:17.950677 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc8c92-22f2-48d6-8cc6-c98909546d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:17.951305 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc8c92-22f2-48d6-8cc6-c98909546d26 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:17.951581 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc8c92-22f2-48d6-8cc6-c98909546d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:17.951854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc8c92-22f2-48d6-8cc6-c98909546d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:17.961033 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc8c92-22f2-48d6-8cc6-c98909546d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:17.962470 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc8c92-22f2-48d6-8cc6-c98909546d26 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:17.971827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1326/2647] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:18.261136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be072c-e920-4c33-bc8b-4fe6106c0660 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:18.262595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-97be072c-e920-4c33-bc8b-4fe6106c0660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:18.275630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97be072c-e920-4c33-bc8b-4fe6106c0660 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:18.286075 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:18.286824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:18.287147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:18.287361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] PATH_INFO: `/v3/projects/353883808ec94809931fe6cbf22df097` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:18.288728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=353883808ec94809931fe6cbf22df097)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:18.290602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:18.340149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '353883808ec94809931fe6cbf22df097', 'request_id': 'req-64db5b74-2184-4f3c-9625-d5b86c97c214'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:18.346820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '353883808ec94809931fe6cbf22df097', 'request_id': 'req-64db5b74-2184-4f3c-9625-d5b86c97c214'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:18.354680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '353883808ec94809931fe6cbf22df097', 'request_id': 'req-64db5b74-2184-4f3c-9625-d5b86c97c214'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:18.365635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] The token cache is being invalidate because project 353883808ec94809931fe6cbf22df097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:11:18.365921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:18.366591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-64db5b74-2184-4f3c-9625-d5b86c97c214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:18.368115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1322/2648] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:11:18 2022] DELETE /identity/v3/projects/353883808ec94809931fe6cbf22df097 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:18.373324 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881afa9-2af1-4bc6-a433-f601de309e78 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:18.373613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881afa9-2af1-4bc6-a433-f601de309e78 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:18.373890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881afa9-2af1-4bc6-a433-f601de309e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:18.395692 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-b881afa9-2af1-4bc6-a433-f601de309e78 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:18.396965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b881afa9-2af1-4bc6-a433-f601de309e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:18.465662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1327/2649] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:18.473772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beec92-3bc8-49f2-96f3-a08014ef0ab7 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:18.474057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beec92-3bc8-49f2-96f3-a08014ef0ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:18.474332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23beec92-3bc8-49f2-96f3-a08014ef0ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:18.495274 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-23beec92-3bc8-49f2-96f3-a08014ef0ab7 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:18.496566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-23beec92-3bc8-49f2-96f3-a08014ef0ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:18.504739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1323/2650] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:18.509355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881afa9-2af1-4bc6-a433-f601de309e78 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:18.520604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881afa9-2af1-4bc6-a433-f601de309e78 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:18.528672 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f446d77-7a59-45d6-b914-e57ab4c263da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:18.529618 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f446d77-7a59-45d6-b914-e57ab4c263da admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:18.530017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f446d77-7a59-45d6-b914-e57ab4c263da admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:18.530308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f446d77-7a59-45d6-b914-e57ab4c263da admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:18.531327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f446d77-7a59-45d6-b914-e57ab4c263da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:11:18.531654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f446d77-7a59-45d6-b914-e57ab4c263da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:18.533842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f446d77-7a59-45d6-b914-e57ab4c263da admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:18.549624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1328/2651] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:11:18.554403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23beec92-3bc8-49f2-96f3-a08014ef0ab7 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:18.564197 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23beec92-3bc8-49f2-96f3-a08014ef0ab7 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:18.570361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034644e4-185a-41e3-8eeb-1620896caf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:18.570969 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034644e4-185a-41e3-8eeb-1620896caf82 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:18.571247 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034644e4-185a-41e3-8eeb-1620896caf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:18.571515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034644e4-185a-41e3-8eeb-1620896caf82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:18.572171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034644e4-185a-41e3-8eeb-1620896caf82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:18.574142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034644e4-185a-41e3-8eeb-1620896caf82 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:18.603327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1324/2652] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:11:18 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:18.607667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f446d77-7a59-45d6-b914-e57ab4c263da admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:18.618964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f446d77-7a59-45d6-b914-e57ab4c263da admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:18.684412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935e7c6-604d-43be-8723-55fd7ad29b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:18.684412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935e7c6-604d-43be-8723-55fd7ad29b97 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:18.684838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935e7c6-604d-43be-8723-55fd7ad29b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:18.684940 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9935e7c6-604d-43be-8723-55fd7ad29b97 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:18.685643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935e7c6-604d-43be-8723-55fd7ad29b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:18.687594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9935e7c6-604d-43be-8723-55fd7ad29b97 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:18.710101 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1329/2653] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:11:18 2022] POST /identity/v3/users => generated 283 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:18.714256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034644e4-185a-41e3-8eeb-1620896caf82 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:18.724632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034644e4-185a-41e3-8eeb-1620896caf82 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:18.729555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9376d47-0061-4e32-818a-fedfd04f4f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:18.730368 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9376d47-0061-4e32-818a-fedfd04f4f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:18.730649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9376d47-0061-4e32-818a-fedfd04f4f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:18.730921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9376d47-0061-4e32-818a-fedfd04f4f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:18.731654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9376d47-0061-4e32-818a-fedfd04f4f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:18.733308 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9376d47-0061-4e32-818a-fedfd04f4f07 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:18.744554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1325/2654] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:18.748943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935e7c6-604d-43be-8723-55fd7ad29b97 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:18.759934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9935e7c6-604d-43be-8723-55fd7ad29b97 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:18.765062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e373e-57a2-45fa-b173-8a4be2d7d249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:18.765684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e373e-57a2-45fa-b173-8a4be2d7d249 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:18.765965 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e373e-57a2-45fa-b173-8a4be2d7d249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:18.766239 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e373e-57a2-45fa-b173-8a4be2d7d249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:18.766943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e373e-57a2-45fa-b173-8a4be2d7d249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:18.768648 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e373e-57a2-45fa-b173-8a4be2d7d249 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:18.780048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1330/2655] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:18.784176 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9376d47-0061-4e32-818a-fedfd04f4f07 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:18.794670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9376d47-0061-4e32-818a-fedfd04f4f07 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:18.799529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604e0f1-a65d-4cbe-9f66-e12372902816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:18.800212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604e0f1-a65d-4cbe-9f66-e12372902816 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:18.800600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604e0f1-a65d-4cbe-9f66-e12372902816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:18.800878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604e0f1-a65d-4cbe-9f66-e12372902816 admin admin] PATH_INFO: `/v3/projects/1595ebf8e8c642aea162ea2bf51a486d/users/6ce261758996420ba4366169528e2b41/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:18.818377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604e0f1-a65d-4cbe-9f66-e12372902816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1595ebf8e8c642aea162ea2bf51a486d, user_id=6ce261758996420ba4366169528e2b41, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:18.821233 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604e0f1-a65d-4cbe-9f66-e12372902816 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:18.835639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:18.842481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1326/2656] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:18 2022] PUT /identity/v3/projects/1595ebf8e8c642aea162ea2bf51a486d/users/6ce261758996420ba4366169528e2b41/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:18.855238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f2817-10f1-40c1-a885-2d7b99cafe3a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:18.855525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f2817-10f1-40c1-a885-2d7b99cafe3a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:18.855800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f2817-10f1-40c1-a885-2d7b99cafe3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:18.884640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-879f2817-10f1-40c1-a885-2d7b99cafe3a None None] MFA Rules not processed for user `6ce261758996420ba4366169528e2b41`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:18.885793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-879f2817-10f1-40c1-a885-2d7b99cafe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:18.912040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:18.959397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1331/2657] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:18.970809 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604e0f1-a65d-4cbe-9f66-e12372902816 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:18.972903 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d604e0f1-a65d-4cbe-9f66-e12372902816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:19.057660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604e0f1-a65d-4cbe-9f66-e12372902816 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:19.062238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bea96-619f-467d-b0f6-ae49e6bc4af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:19.062869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bea96-619f-467d-b0f6-ae49e6bc4af4 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:19.063148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bea96-619f-467d-b0f6-ae49e6bc4af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:19.063422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bea96-619f-467d-b0f6-ae49e6bc4af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:19.073669 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bea96-619f-467d-b0f6-ae49e6bc4af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:19.075099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bea96-619f-467d-b0f6-ae49e6bc4af4 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:19.084487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1327/2658] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:20.946114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872e1f1-9f5d-47d8-b84b-a76dd65cf86c None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:20.946764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872e1f1-9f5d-47d8-b84b-a76dd65cf86c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:20.946764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872e1f1-9f5d-47d8-b84b-a76dd65cf86c None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:20.948405 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1332/2659] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:11:20.954460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ef9b5-8d18-40e9-8a38-b4b5ec554744 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:20.954963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ef9b5-8d18-40e9-8a38-b4b5ec554744 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:20.955067 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ef9b5-8d18-40e9-8a38-b4b5ec554744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:20.975237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-5f6ef9b5-8d18-40e9-8a38-b4b5ec554744 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:20.976530 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5f6ef9b5-8d18-40e9-8a38-b4b5ec554744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:21.053382 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1328/2660] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:11:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:21.066575 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872e1f1-9f5d-47d8-b84b-a76dd65cf86c None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:21.068338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3872e1f1-9f5d-47d8-b84b-a76dd65cf86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:21.130126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872e1f1-9f5d-47d8-b84b-a76dd65cf86c None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:21.134215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf960c9d-dc3e-4f87-afe0-0d44d5d3cb26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:21.134839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf960c9d-dc3e-4f87-afe0-0d44d5d3cb26 service glance] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:21.135115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf960c9d-dc3e-4f87-afe0-0d44d5d3cb26 service glance] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:21.135388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf960c9d-dc3e-4f87-afe0-0d44d5d3cb26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:21.146685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf960c9d-dc3e-4f87-afe0-0d44d5d3cb26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:21.148162 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf960c9d-dc3e-4f87-afe0-0d44d5d3cb26 service glance] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:21.162494 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1333/2661] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:21.236899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd214204-5ed8-4a2a-acd3-af8fe775a954 None None] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:21.237171 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd214204-5ed8-4a2a-acd3-af8fe775a954 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:21.237436 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd214204-5ed8-4a2a-acd3-af8fe775a954 None None] PATH_INFO: `/` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:21.239194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1329/2662] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:11:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 01 12:11:21.244507 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959821-023a-4816-bc20-3db2d16fd117 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:21.244796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959821-023a-4816-bc20-3db2d16fd117 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:21.245236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6959821-023a-4816-bc20-3db2d16fd117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:21.267296 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-a6959821-023a-4816-bc20-3db2d16fd117 None None] MFA Rules not processed for user `8dfbda2838b242558c885f913fafb5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:21.268556 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a6959821-023a-4816-bc20-3db2d16fd117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:21.326959 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1334/2663] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:11:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:21.339239 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd214204-5ed8-4a2a-acd3-af8fe775a954 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:21.341617 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fd214204-5ed8-4a2a-acd3-af8fe775a954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:21.357770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd214204-5ed8-4a2a-acd3-af8fe775a954 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:21.362213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc762a3d-e48a-4ce2-9736-448217bad921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:21.362803 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc762a3d-e48a-4ce2-9736-448217bad921 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:21.363083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc762a3d-e48a-4ce2-9736-448217bad921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:21.363378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc762a3d-e48a-4ce2-9736-448217bad921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:21.371810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc762a3d-e48a-4ce2-9736-448217bad921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:21.373284 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc762a3d-e48a-4ce2-9736-448217bad921 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:21.381232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1330/2664] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:21.468135 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7990729-7c75-4b58-a3cb-cb7c49ce8903 None None] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:21.468457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7990729-7c75-4b58-a3cb-cb7c49ce8903 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:21.468714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7990729-7c75-4b58-a3cb-cb7c49ce8903 None None] PATH_INFO: `/` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:21.470219 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1335/2665] 10.222.0.49 () {54 vars in 921 bytes} [Mon Aug 1 12:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 01 12:11:21.476354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34464817-b3c7-4a03-bae6-5290242d8d68 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:21.476835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34464817-b3c7-4a03-bae6-5290242d8d68 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:21.477460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34464817-b3c7-4a03-bae6-5290242d8d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:21.503383 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-34464817-b3c7-4a03-bae6-5290242d8d68 None None] MFA Rules not processed for user `db3ff1b21c784e8cb5bffcb79f230d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:21.504633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-34464817-b3c7-4a03-bae6-5290242d8d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:21.516718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1331/2666] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Aug 1 12:11:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:11:21.528178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7990729-7c75-4b58-a3cb-cb7c49ce8903 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:21.530327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f7990729-7c75-4b58-a3cb-cb7c49ce8903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:21.546181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7990729-7c75-4b58-a3cb-cb7c49ce8903 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:21.550369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb54af-e617-45d2-9306-02bcca9fd8fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:21.551002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb54af-e617-45d2-9306-02bcca9fd8fb service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:21.551311 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb54af-e617-45d2-9306-02bcca9fd8fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:21.551577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb54af-e617-45d2-9306-02bcca9fd8fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:21.563102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb54af-e617-45d2-9306-02bcca9fd8fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:21.564643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb54af-e617-45d2-9306-02bcca9fd8fb service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:21.578056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1336/2667] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:23.284472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0611c-2477-4b7e-80c8-d6d21e66b5df None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:23.284944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0611c-2477-4b7e-80c8-d6d21e66b5df None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:23.285058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0611c-2477-4b7e-80c8-d6d21e66b5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:23.306326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a1b0611c-2477-4b7e-80c8-d6d21e66b5df None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:23.307226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a1b0611c-2477-4b7e-80c8-d6d21e66b5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:23.374324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:23.383850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1332/2668] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:11:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:23.397410 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb54af-e617-45d2-9306-02bcca9fd8fb service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:23.398895 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a4bb54af-e617-45d2-9306-02bcca9fd8fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:23.476564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:23.481805 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb54af-e617-45d2-9306-02bcca9fd8fb service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:23.487027 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec7c97-0978-4545-bcf0-443166adc1c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:23.487654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec7c97-0978-4545-bcf0-443166adc1c5 service swift] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:23.487935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec7c97-0978-4545-bcf0-443166adc1c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:23.488210 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec7c97-0978-4545-bcf0-443166adc1c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:23.497868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec7c97-0978-4545-bcf0-443166adc1c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:23.500910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec7c97-0978-4545-bcf0-443166adc1c5 service swift] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:23.510313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1337/2669] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:11:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 01 12:11:27.781697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0611c-2477-4b7e-80c8-d6d21e66b5df None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:27.853605 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0611c-2477-4b7e-80c8-d6d21e66b5df None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:27.858796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:27.859471 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:27.859754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:27.860029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] PATH_INFO: `/v3/users/6ce261758996420ba4366169528e2b41` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:27.861275 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ce261758996420ba4366169528e2b41)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:27.863391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:27.925107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ce261758996420ba4366169528e2b41'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:27.938236 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ce261758996420ba4366169528e2b41'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:27.945432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ce261758996420ba4366169528e2b41'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:27.946173 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ce261758996420ba4366169528e2b41'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:27.946764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ce261758996420ba4366169528e2b41'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:27.966314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1333/2670] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:11:27 2022] DELETE /identity/v3/users/6ce261758996420ba4366169528e2b41 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:27.967883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df740dd-a918-45e4-ac32-9bbf51b0f079 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:27.968385 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df740dd-a918-45e4-ac32-9bbf51b0f079 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:27.968499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df740dd-a918-45e4-ac32-9bbf51b0f079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:27.998861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3df740dd-a918-45e4-ac32-9bbf51b0f079 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:28.000716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3df740dd-a918-45e4-ac32-9bbf51b0f079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:28.074554 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1338/2671] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:11:28.088460 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:28.090257 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:28.200174 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1813a89-fc62-4c4e-ab7c-8dea60293279 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:28.215172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372f0e3-113d-47f2-ac4a-bd62acfbcb59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:28.217830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372f0e3-113d-47f2-ac4a-bd62acfbcb59 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:28.217830 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372f0e3-113d-47f2-ac4a-bd62acfbcb59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:28.219863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372f0e3-113d-47f2-ac4a-bd62acfbcb59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:28.269936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372f0e3-113d-47f2-ac4a-bd62acfbcb59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:28.272087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372f0e3-113d-47f2-ac4a-bd62acfbcb59 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:28.288047 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1334/2672] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:28.673904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df740dd-a918-45e4-ac32-9bbf51b0f079 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:28.675700 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3df740dd-a918-45e4-ac32-9bbf51b0f079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:28.699198 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df740dd-a918-45e4-ac32-9bbf51b0f079 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:28.704908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:28.705661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:28.705943 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:28.706294 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] PATH_INFO: `/v3/projects/1595ebf8e8c642aea162ea2bf51a486d` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:28.707636 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1595ebf8e8c642aea162ea2bf51a486d)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:28.709559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:28.770379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1595ebf8e8c642aea162ea2bf51a486d', 'request_id': 'req-26105f3b-8171-47a3-9e32-ee8c80f448d1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:28.781914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1595ebf8e8c642aea162ea2bf51a486d', 'request_id': 'req-26105f3b-8171-47a3-9e32-ee8c80f448d1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:28.791295 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1595ebf8e8c642aea162ea2bf51a486d', 'request_id': 'req-26105f3b-8171-47a3-9e32-ee8c80f448d1'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:28.803512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] The token cache is being invalidate because project 1595ebf8e8c642aea162ea2bf51a486d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:11:28.803803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:28.804514 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-26105f3b-8171-47a3-9e32-ee8c80f448d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:28.809747 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1339/2673] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:11:28 2022] DELETE /identity/v3/projects/1595ebf8e8c642aea162ea2bf51a486d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:28.811665 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4371e3-36c3-4ec6-8429-638a9a3dba87 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:28.812437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4371e3-36c3-4ec6-8429-638a9a3dba87 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:28.812437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4371e3-36c3-4ec6-8429-638a9a3dba87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:28.834136 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-8c4371e3-36c3-4ec6-8429-638a9a3dba87 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:28.835487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-8c4371e3-36c3-4ec6-8429-638a9a3dba87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:28.863093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:28.923465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1335/2674] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:28.931504 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4ef11-6fbe-4cb5-9add-24c54ac1d9ff None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:28.931791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4ef11-6fbe-4cb5-9add-24c54ac1d9ff None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:28.932069 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4ef11-6fbe-4cb5-9add-24c54ac1d9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:28.952864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-69d4ef11-6fbe-4cb5-9add-24c54ac1d9ff None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:28.954103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-69d4ef11-6fbe-4cb5-9add-24c54ac1d9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:28.962861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1340/2675] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:28.967137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4371e3-36c3-4ec6-8429-638a9a3dba87 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:28.977894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4371e3-36c3-4ec6-8429-638a9a3dba87 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:28.982652 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ffb4d-e943-44ca-8234-23f306770e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:28.983289 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ffb4d-e943-44ca-8234-23f306770e64 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:28.983581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ffb4d-e943-44ca-8234-23f306770e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:28.983857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ffb4d-e943-44ca-8234-23f306770e64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:28.984615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ffb4d-e943-44ca-8234-23f306770e64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:11:28.984926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ffb4d-e943-44ca-8234-23f306770e64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:28.986554 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ffb4d-e943-44ca-8234-23f306770e64 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:29.004340 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1336/2676] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:11:29.004577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4ef11-6fbe-4cb5-9add-24c54ac1d9ff None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:29.014560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4ef11-6fbe-4cb5-9add-24c54ac1d9ff None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:29.020177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cece7-c219-46a3-a437-67e277fea113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:29.020843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cece7-c219-46a3-a437-67e277fea113 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:29.021205 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cece7-c219-46a3-a437-67e277fea113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:29.021438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931cece7-c219-46a3-a437-67e277fea113 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:29.022108 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931cece7-c219-46a3-a437-67e277fea113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:29.024359 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931cece7-c219-46a3-a437-67e277fea113 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:29.064329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1341/2677] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:11:29 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:11:29.066553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ffb4d-e943-44ca-8234-23f306770e64 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:29.077688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ffb4d-e943-44ca-8234-23f306770e64 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:29.148454 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b395a-f68e-437f-a986-732caf57a3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:29.149106 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b395a-f68e-437f-a986-732caf57a3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:29.149381 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b395a-f68e-437f-a986-732caf57a3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:29.149649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b395a-f68e-437f-a986-732caf57a3a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:29.150316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b395a-f68e-437f-a986-732caf57a3a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:29.152282 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b395a-f68e-437f-a986-732caf57a3a0 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:29.175525 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1337/2678] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:11:29 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:29.179726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cece7-c219-46a3-a437-67e277fea113 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:29.191659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931cece7-c219-46a3-a437-67e277fea113 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:29.197208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbecb3-53b8-4e34-8919-159d675251cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:29.197851 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbecb3-53b8-4e34-8919-159d675251cc admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:29.198206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbecb3-53b8-4e34-8919-159d675251cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:29.198622 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbecb3-53b8-4e34-8919-159d675251cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:29.199365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbecb3-53b8-4e34-8919-159d675251cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:29.201205 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbecb3-53b8-4e34-8919-159d675251cc admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:29.213978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1342/2679] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:29.217995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b395a-f68e-437f-a986-732caf57a3a0 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:29.231351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b395a-f68e-437f-a986-732caf57a3a0 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:29.236121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b7044-2729-47e6-a993-7d829d20c46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:29.236799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b7044-2729-47e6-a993-7d829d20c46b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:29.237079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b7044-2729-47e6-a993-7d829d20c46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:29.237348 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b7044-2729-47e6-a993-7d829d20c46b admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:29.238052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b7044-2729-47e6-a993-7d829d20c46b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:29.240046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b7044-2729-47e6-a993-7d829d20c46b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:29.252265 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1338/2680] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:11:29.256582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbecb3-53b8-4e34-8919-159d675251cc admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:29.268878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbecb3-53b8-4e34-8919-159d675251cc admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:29.273935 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18499ba6-d155-40e6-a5c5-008bb321a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:29.274762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18499ba6-d155-40e6-a5c5-008bb321a553 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:29.275171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18499ba6-d155-40e6-a5c5-008bb321a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:29.275518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18499ba6-d155-40e6-a5c5-008bb321a553 admin admin] PATH_INFO: `/v3/projects/62ee3be01cb64bcf8eae901cdced8f9a/users/fbba271214c64331833772f0eaa3bebf/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:29.295278 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18499ba6-d155-40e6-a5c5-008bb321a553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ee3be01cb64bcf8eae901cdced8f9a, user_id=fbba271214c64331833772f0eaa3bebf, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:29.299745 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18499ba6-d155-40e6-a5c5-008bb321a553 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:29.312902 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1343/2681] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:29 2022] PUT /identity/v3/projects/62ee3be01cb64bcf8eae901cdced8f9a/users/fbba271214c64331833772f0eaa3bebf/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:11:29.323753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd342f-cd6b-47db-bc1c-7caf9df16aae None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:29.324172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd342f-cd6b-47db-bc1c-7caf9df16aae None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:29.324563 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd342f-cd6b-47db-bc1c-7caf9df16aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:29.363262 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-39fd342f-cd6b-47db-bc1c-7caf9df16aae None None] MFA Rules not processed for user `fbba271214c64331833772f0eaa3bebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:29.364930 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-39fd342f-cd6b-47db-bc1c-7caf9df16aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:29.490486 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1339/2682] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:29.498728 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18499ba6-d155-40e6-a5c5-008bb321a553 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:29.500907 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-18499ba6-d155-40e6-a5c5-008bb321a553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:29.517868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:29.654497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18499ba6-d155-40e6-a5c5-008bb321a553 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:29.661545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb4263-1e30-4a3d-9f89-c034218fa999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:29.662191 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb4263-1e30-4a3d-9f89-c034218fa999 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:29.662471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb4263-1e30-4a3d-9f89-c034218fa999 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:29.662746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb4263-1e30-4a3d-9f89-c034218fa999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:29.674501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb4263-1e30-4a3d-9f89-c034218fa999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:29.676401 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb4263-1e30-4a3d-9f89-c034218fa999 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:29.689181 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1344/2683] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:29.956237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd342f-cd6b-47db-bc1c-7caf9df16aae None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:29.984566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:30.027498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd342f-cd6b-47db-bc1c-7caf9df16aae None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:30.032044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:30.032745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:30.033027 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:30.033301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] PATH_INFO: `/v3/users/fbba271214c64331833772f0eaa3bebf` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:30.034662 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbba271214c64331833772f0eaa3bebf)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:30.036690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:30.089987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbba271214c64331833772f0eaa3bebf'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:30.103448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbba271214c64331833772f0eaa3bebf'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:30.112779 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbba271214c64331833772f0eaa3bebf'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:30.113957 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbba271214c64331833772f0eaa3bebf'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:30.114566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbba271214c64331833772f0eaa3bebf'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:30.139599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1340/2684] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:11:29 2022] DELETE /identity/v3/users/fbba271214c64331833772f0eaa3bebf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:30.142248 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945146a-847e-4a58-912e-d5154140d409 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:30.143515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945146a-847e-4a58-912e-d5154140d409 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:30.144011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945146a-847e-4a58-912e-d5154140d409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:30.169786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7945146a-847e-4a58-912e-d5154140d409 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:30.172271 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7945146a-847e-4a58-912e-d5154140d409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:30.244421 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1345/2685] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:11:30.254595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:30.256181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:30.342452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbaa56-97fa-4c8a-8e89-e78faa3b26ac admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:30.346107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ca685-9672-4681-b0da-487213719fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:30.346745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ca685-9672-4681-b0da-487213719fdd service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:30.347029 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ca685-9672-4681-b0da-487213719fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:30.347303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ca685-9672-4681-b0da-487213719fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:30.359452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ca685-9672-4681-b0da-487213719fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:30.360993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ca685-9672-4681-b0da-487213719fdd service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:30.372344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1341/2686] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:30.691483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945146a-847e-4a58-912e-d5154140d409 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:30.693406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7945146a-847e-4a58-912e-d5154140d409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:30.713873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945146a-847e-4a58-912e-d5154140d409 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:30.719380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:30.720092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:30.720407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:30.720684 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] PATH_INFO: `/v3/projects/62ee3be01cb64bcf8eae901cdced8f9a` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:30.721936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62ee3be01cb64bcf8eae901cdced8f9a)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:30.724126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:30.804070 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62ee3be01cb64bcf8eae901cdced8f9a', 'request_id': 'req-114b583a-1a32-4296-8d73-3b7ac3276b55'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:30.815834 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62ee3be01cb64bcf8eae901cdced8f9a', 'request_id': 'req-114b583a-1a32-4296-8d73-3b7ac3276b55'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:30.834987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62ee3be01cb64bcf8eae901cdced8f9a', 'request_id': 'req-114b583a-1a32-4296-8d73-3b7ac3276b55'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:30.856197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] The token cache is being invalidate because project 62ee3be01cb64bcf8eae901cdced8f9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:11:30.856197 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:30.857644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-114b583a-1a32-4296-8d73-3b7ac3276b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:30.859632 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1346/2687] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:11:30 2022] DELETE /identity/v3/projects/62ee3be01cb64bcf8eae901cdced8f9a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:30.868695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c525a-be48-4023-8ce0-27f39b4df6ab None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:30.869108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c525a-be48-4023-8ce0-27f39b4df6ab None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:30.869491 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c525a-be48-4023-8ce0-27f39b4df6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:30.893106 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-af2c525a-be48-4023-8ce0-27f39b4df6ab None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:30.894354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-af2c525a-be48-4023-8ce0-27f39b4df6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:30.930667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:30.960490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1342/2688] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:30.970221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d48386-5ec2-4268-8b95-9eab480dc3e7 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:30.970526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d48386-5ec2-4268-8b95-9eab480dc3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:30.970793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d48386-5ec2-4268-8b95-9eab480dc3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:30.993768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-e2d48386-5ec2-4268-8b95-9eab480dc3e7 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:30.995122 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e2d48386-5ec2-4268-8b95-9eab480dc3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:31.004813 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1347/2689] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:31.009250 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c525a-be48-4023-8ce0-27f39b4df6ab None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.019438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c525a-be48-4023-8ce0-27f39b4df6ab None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.025134 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd8db9-dfc2-4bab-98fb-42c73967099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.025758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd8db9-dfc2-4bab-98fb-42c73967099c admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.026037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd8db9-dfc2-4bab-98fb-42c73967099c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.026369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd8db9-dfc2-4bab-98fb-42c73967099c admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.027078 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd8db9-dfc2-4bab-98fb-42c73967099c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:11:31.027392 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd8db9-dfc2-4bab-98fb-42c73967099c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.029052 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dd8db9-dfc2-4bab-98fb-42c73967099c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.041402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1343/2690] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:11:31.045982 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d48386-5ec2-4268-8b95-9eab480dc3e7 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.055861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d48386-5ec2-4268-8b95-9eab480dc3e7 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.062236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ce50f-117e-48c8-8686-894bd2ae76dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.062849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ce50f-117e-48c8-8686-894bd2ae76dc admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.063125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ce50f-117e-48c8-8686-894bd2ae76dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.063396 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ce50f-117e-48c8-8686-894bd2ae76dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.064061 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ce50f-117e-48c8-8686-894bd2ae76dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.066092 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ce50f-117e-48c8-8686-894bd2ae76dc admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.077909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:31.098816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1348/2691] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:11:31 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:31.103147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd8db9-dfc2-4bab-98fb-42c73967099c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.113933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd8db9-dfc2-4bab-98fb-42c73967099c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.201043 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f59e4b-0195-49e1-aefb-3831319ff21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.201702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f59e4b-0195-49e1-aefb-3831319ff21f admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.201982 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f59e4b-0195-49e1-aefb-3831319ff21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.202254 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f59e4b-0195-49e1-aefb-3831319ff21f admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.202927 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f59e4b-0195-49e1-aefb-3831319ff21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.204908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f59e4b-0195-49e1-aefb-3831319ff21f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.229332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1344/2692] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:11:31 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:31.236531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ce50f-117e-48c8-8686-894bd2ae76dc admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.252142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ce50f-117e-48c8-8686-894bd2ae76dc admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.258165 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b8776-596e-4fe6-b71e-743a31596a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.258822 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b8776-596e-4fe6-b71e-743a31596a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.259099 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b8776-596e-4fe6-b71e-743a31596a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.259364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b8776-596e-4fe6-b71e-743a31596a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.260062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b8776-596e-4fe6-b71e-743a31596a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.261864 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b8776-596e-4fe6-b71e-743a31596a4e admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.272722 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1349/2693] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:31.277091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f59e4b-0195-49e1-aefb-3831319ff21f admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.287504 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f59e4b-0195-49e1-aefb-3831319ff21f admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.292770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c3d2-99c9-4c8c-83ee-e31670f09312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.293386 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c3d2-99c9-4c8c-83ee-e31670f09312 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.293664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c3d2-99c9-4c8c-83ee-e31670f09312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.293941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c3d2-99c9-4c8c-83ee-e31670f09312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.294633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c3d2-99c9-4c8c-83ee-e31670f09312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.296398 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c3d2-99c9-4c8c-83ee-e31670f09312 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.307894 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1345/2694] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:31.311973 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b8776-596e-4fe6-b71e-743a31596a4e admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.323250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b8776-596e-4fe6-b71e-743a31596a4e admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.328080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3a19-04ef-461a-9d22-3aae369b66eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.328784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3a19-04ef-461a-9d22-3aae369b66eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.329059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3a19-04ef-461a-9d22-3aae369b66eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.329323 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc3a19-04ef-461a-9d22-3aae369b66eb admin admin] PATH_INFO: `/v3/projects/3956836e33ce4c7daf8b54bc64c39616/users/db5f9f49c7ef4a51b1635f21120e6957/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.344158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc3a19-04ef-461a-9d22-3aae369b66eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3956836e33ce4c7daf8b54bc64c39616, user_id=db5f9f49c7ef4a51b1635f21120e6957, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.346820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc3a19-04ef-461a-9d22-3aae369b66eb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.361573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1350/2695] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:31 2022] PUT /identity/v3/projects/3956836e33ce4c7daf8b54bc64c39616/users/db5f9f49c7ef4a51b1635f21120e6957/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:31.372768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f2171-ff49-4e57-9e81-e86d676f23b8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.373050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f2171-ff49-4e57-9e81-e86d676f23b8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.373322 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f2171-ff49-4e57-9e81-e86d676f23b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.404042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-524f2171-ff49-4e57-9e81-e86d676f23b8 None None] MFA Rules not processed for user `db5f9f49c7ef4a51b1635f21120e6957`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:31.405318 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-524f2171-ff49-4e57-9e81-e86d676f23b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:31.473599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1346/2696] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:31.478406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3a19-04ef-461a-9d22-3aae369b66eb admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.542338 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc3a19-04ef-461a-9d22-3aae369b66eb admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.548364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c73e1-cfcf-46bc-9d9b-b078eafa7cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.549008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c73e1-cfcf-46bc-9d9b-b078eafa7cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.549287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c73e1-cfcf-46bc-9d9b-b078eafa7cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.549584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c73e1-cfcf-46bc-9d9b-b078eafa7cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.550264 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c73e1-cfcf-46bc-9d9b-b078eafa7cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.552420 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c73e1-cfcf-46bc-9d9b-b078eafa7cc2 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.580854 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1351/2697] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:11:31 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:31.585008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f2171-ff49-4e57-9e81-e86d676f23b8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.596553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f2171-ff49-4e57-9e81-e86d676f23b8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.630738 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:31.658570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa79c87-e082-4511-a063-cb756a3b0d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.659196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa79c87-e082-4511-a063-cb756a3b0d16 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.659476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa79c87-e082-4511-a063-cb756a3b0d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.659747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa79c87-e082-4511-a063-cb756a3b0d16 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.660454 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa79c87-e082-4511-a063-cb756a3b0d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.662378 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa79c87-e082-4511-a063-cb756a3b0d16 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.687721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1347/2698] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:11:31 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:31.691888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c73e1-cfcf-46bc-9d9b-b078eafa7cc2 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.702631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c73e1-cfcf-46bc-9d9b-b078eafa7cc2 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.707839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be6668-892e-499e-a089-1942309b54d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.708489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be6668-892e-499e-a089-1942309b54d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.708774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be6668-892e-499e-a089-1942309b54d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.709047 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be6668-892e-499e-a089-1942309b54d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.709751 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be6668-892e-499e-a089-1942309b54d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.711404 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be6668-892e-499e-a089-1942309b54d0 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.722389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1352/2699] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:11:31.726562 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa79c87-e082-4511-a063-cb756a3b0d16 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.737080 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa79c87-e082-4511-a063-cb756a3b0d16 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.742501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52100450-cb45-4cd9-9b8e-52f2b76116d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.743181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52100450-cb45-4cd9-9b8e-52f2b76116d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.743485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52100450-cb45-4cd9-9b8e-52f2b76116d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.743904 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52100450-cb45-4cd9-9b8e-52f2b76116d5 admin admin] PATH_INFO: `/v3/projects/b7a9b56b7e5546078c920e39bcf8963e/users/accc54ce657f406fac04f30cd8e0bda8/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.760600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52100450-cb45-4cd9-9b8e-52f2b76116d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a9b56b7e5546078c920e39bcf8963e, user_id=accc54ce657f406fac04f30cd8e0bda8, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.763357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52100450-cb45-4cd9-9b8e-52f2b76116d5 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.782675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1348/2700] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:31 2022] PUT /identity/v3/projects/b7a9b56b7e5546078c920e39bcf8963e/users/accc54ce657f406fac04f30cd8e0bda8/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:31.786875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be6668-892e-499e-a089-1942309b54d0 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.797887 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be6668-892e-499e-a089-1942309b54d0 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.870040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a699fc99-fb46-477c-b5b2-4f941ebaa146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.870704 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699fc99-fb46-477c-b5b2-4f941ebaa146 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.871077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699fc99-fb46-477c-b5b2-4f941ebaa146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.871515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a699fc99-fb46-477c-b5b2-4f941ebaa146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.872244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699fc99-fb46-477c-b5b2-4f941ebaa146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.873916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a699fc99-fb46-477c-b5b2-4f941ebaa146 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.887814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1353/2701] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:31.889079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52100450-cb45-4cd9-9b8e-52f2b76116d5 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:31.900035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52100450-cb45-4cd9-9b8e-52f2b76116d5 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:31.905228 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3eb87-f966-4987-a80c-e69c1150dbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:31.905891 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3eb87-f966-4987-a80c-e69c1150dbc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.906175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3eb87-f966-4987-a80c-e69c1150dbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.906448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3eb87-f966-4987-a80c-e69c1150dbc6 admin admin] PATH_INFO: `/v3/projects/b7a9b56b7e5546078c920e39bcf8963e/users/accc54ce657f406fac04f30cd8e0bda8/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.908929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3eb87-f966-4987-a80c-e69c1150dbc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a9b56b7e5546078c920e39bcf8963e, user_id=accc54ce657f406fac04f30cd8e0bda8, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:31.912320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3eb87-f966-4987-a80c-e69c1150dbc6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:31.923199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1349/2702] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:31 2022] PUT /identity/v3/projects/b7a9b56b7e5546078c920e39bcf8963e/users/accc54ce657f406fac04f30cd8e0bda8/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:31.933997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254afec4-d2d8-415e-98cf-1ac6be557596 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:31.934388 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254afec4-d2d8-415e-98cf-1ac6be557596 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:31.934714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254afec4-d2d8-415e-98cf-1ac6be557596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:31.965024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-254afec4-d2d8-415e-98cf-1ac6be557596 None None] MFA Rules not processed for user `accc54ce657f406fac04f30cd8e0bda8`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:31.966282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-254afec4-d2d8-415e-98cf-1ac6be557596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:31.985037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:32.041013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1354/2703] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:32.052415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3eb87-f966-4987-a80c-e69c1150dbc6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:32.054184 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d1b3eb87-f966-4987-a80c-e69c1150dbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:32.115742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3eb87-f966-4987-a80c-e69c1150dbc6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:32.119406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59917c88-cbfd-4bd0-b499-f784cc999ce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:32.120063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917c88-cbfd-4bd0-b499-f784cc999ce4 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:32.120385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917c88-cbfd-4bd0-b499-f784cc999ce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:32.120663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59917c88-cbfd-4bd0-b499-f784cc999ce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:32.130042 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59917c88-cbfd-4bd0-b499-f784cc999ce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:32.131490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59917c88-cbfd-4bd0-b499-f784cc999ce4 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:32.141351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1350/2704] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:32.726310 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254afec4-d2d8-415e-98cf-1ac6be557596 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:32.795454 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254afec4-d2d8-415e-98cf-1ac6be557596 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:32.800838 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:32.801498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:32.801777 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:32.802051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] PATH_INFO: `/v3/users/db5f9f49c7ef4a51b1635f21120e6957` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:32.803266 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5f9f49c7ef4a51b1635f21120e6957)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:32.805268 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:32.859345 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5f9f49c7ef4a51b1635f21120e6957'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:32.860035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5f9f49c7ef4a51b1635f21120e6957'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:32.869993 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5f9f49c7ef4a51b1635f21120e6957'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:32.883221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5f9f49c7ef4a51b1635f21120e6957'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:32.890824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5f9f49c7ef4a51b1635f21120e6957'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:32.896938 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1355/2705] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:11:32 2022] DELETE /identity/v3/users/db5f9f49c7ef4a51b1635f21120e6957 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:32.901675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb40a2e-84a0-4c07-a4cf-22a18a57542e None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:32.901967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb40a2e-84a0-4c07-a4cf-22a18a57542e None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:32.902246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb40a2e-84a0-4c07-a4cf-22a18a57542e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:32.922319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-2eb40a2e-84a0-4c07-a4cf-22a18a57542e None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:32.923451 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2eb40a2e-84a0-4c07-a4cf-22a18a57542e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:32.989967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1351/2706] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:11:33.000064 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:33.001633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:33.037968 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:33.064423 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae81b7-7f51-41e5-8498-a0748e39fb14 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:33.069231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183e527-364c-4342-abce-bae4aebef1a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:33.069914 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183e527-364c-4342-abce-bae4aebef1a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:33.070214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183e527-364c-4342-abce-bae4aebef1a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:33.070484 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183e527-364c-4342-abce-bae4aebef1a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:33.085445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0183e527-364c-4342-abce-bae4aebef1a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:33.087695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0183e527-364c-4342-abce-bae4aebef1a6 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:33.104224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1356/2707] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:33.379003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb40a2e-84a0-4c07-a4cf-22a18a57542e None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:33.380526 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-2eb40a2e-84a0-4c07-a4cf-22a18a57542e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:33.392518 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb40a2e-84a0-4c07-a4cf-22a18a57542e None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:33.397278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:33.397934 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:33.398211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:33.398489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] PATH_INFO: `/v3/projects/3956836e33ce4c7daf8b54bc64c39616` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:33.399666 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3956836e33ce4c7daf8b54bc64c39616)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:33.401739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:33.434618 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:33.457183 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3956836e33ce4c7daf8b54bc64c39616', 'request_id': 'req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:33.465316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3956836e33ce4c7daf8b54bc64c39616', 'request_id': 'req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:33.475164 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3956836e33ce4c7daf8b54bc64c39616', 'request_id': 'req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:33.485997 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] The token cache is being invalidate because project 3956836e33ce4c7daf8b54bc64c39616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:11:33.486283 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:33.486920 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:33.488840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1352/2708] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:11:33 2022] DELETE /identity/v3/projects/3956836e33ce4c7daf8b54bc64c39616 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:33.493055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183e527-364c-4342-abce-bae4aebef1a6 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:33.503321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183e527-364c-4342-abce-bae4aebef1a6 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:33.565334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:33.565898 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:33.566167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:33.566449 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] PATH_INFO: `/v3/users/accc54ce657f406fac04f30cd8e0bda8` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:33.567804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=accc54ce657f406fac04f30cd8e0bda8)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:33.570146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:33.621085 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'accc54ce657f406fac04f30cd8e0bda8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:33.621828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'accc54ce657f406fac04f30cd8e0bda8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:33.632015 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'accc54ce657f406fac04f30cd8e0bda8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:33.648979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'accc54ce657f406fac04f30cd8e0bda8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:33.656413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-1b996e79-bae6-4ed3-9b84-828c28d00ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'accc54ce657f406fac04f30cd8e0bda8'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:33.662489 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1357/2709] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:11:33 2022] DELETE /identity/v3/users/accc54ce657f406fac04f30cd8e0bda8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:33.946124 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:33.947612 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:33.961707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7502e8-a8fc-4640-99e2-1e8102ea1a7c admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:34.038726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:34.039349 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.039613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.039883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] PATH_INFO: `/v3/projects/b7a9b56b7e5546078c920e39bcf8963e` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.041165 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7a9b56b7e5546078c920e39bcf8963e)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:34.043087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:34.096954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7a9b56b7e5546078c920e39bcf8963e', 'request_id': 'req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:34.103297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7a9b56b7e5546078c920e39bcf8963e', 'request_id': 'req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:34.112761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7a9b56b7e5546078c920e39bcf8963e', 'request_id': 'req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:34.124206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] The token cache is being invalidate because project b7a9b56b7e5546078c920e39bcf8963e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:11:34.124543 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:34.125188 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-42dbcdc8-f180-4f92-b1f8-9002d9842ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:34.126702 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1353/2710] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:11:33 2022] DELETE /identity/v3/projects/b7a9b56b7e5546078c920e39bcf8963e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:34.133551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6582f80-f6c6-4ac0-8296-a063a93f4932 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.134062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6582f80-f6c6-4ac0-8296-a063a93f4932 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.134062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6582f80-f6c6-4ac0-8296-a063a93f4932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.285995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c6582f80-f6c6-4ac0-8296-a063a93f4932 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:34.287228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c6582f80-f6c6-4ac0-8296-a063a93f4932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:34.338206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:34.353650 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1358/2711] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:34.363186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb01602-aea6-4be4-9088-badeb422a22d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.363818 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb01602-aea6-4be4-9088-badeb422a22d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.364103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb01602-aea6-4be4-9088-badeb422a22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.389815 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-3bb01602-aea6-4be4-9088-badeb422a22d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:34.391007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-3bb01602-aea6-4be4-9088-badeb422a22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:34.399139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1354/2712] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:34.403601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6582f80-f6c6-4ac0-8296-a063a93f4932 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:34.413204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6582f80-f6c6-4ac0-8296-a063a93f4932 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:34.417870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5f397-fa59-462d-92ac-6573fc565e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:34.418493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5f397-fa59-462d-92ac-6573fc565e7b admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.418774 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5f397-fa59-462d-92ac-6573fc565e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.419048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5f397-fa59-462d-92ac-6573fc565e7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.420270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5f397-fa59-462d-92ac-6573fc565e7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:11:34.420270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5f397-fa59-462d-92ac-6573fc565e7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:34.421667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5f397-fa59-462d-92ac-6573fc565e7b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:34.433934 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1359/2713] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:11:34.438323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb01602-aea6-4be4-9088-badeb422a22d None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:34.447668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb01602-aea6-4be4-9088-badeb422a22d None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:34.452519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b6395-fc1b-4612-a9de-965eacadddbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:34.453133 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6395-fc1b-4612-a9de-965eacadddbe admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.453405 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6395-fc1b-4612-a9de-965eacadddbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.453673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049b6395-fc1b-4612-a9de-965eacadddbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.454361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b6395-fc1b-4612-a9de-965eacadddbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:34.456261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049b6395-fc1b-4612-a9de-965eacadddbe admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:34.487833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1355/2714] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:11:34 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:34.492460 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5f397-fa59-462d-92ac-6573fc565e7b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:34.503285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5f397-fa59-462d-92ac-6573fc565e7b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:34.566343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae45bec-8292-4e0c-a684-6dde9d6ab01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:34.567002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45bec-8292-4e0c-a684-6dde9d6ab01b admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.567282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45bec-8292-4e0c-a684-6dde9d6ab01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.567663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae45bec-8292-4e0c-a684-6dde9d6ab01b admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.568512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae45bec-8292-4e0c-a684-6dde9d6ab01b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:34.570784 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae45bec-8292-4e0c-a684-6dde9d6ab01b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:34.601929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1360/2715] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:11:34 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:34.606450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b6395-fc1b-4612-a9de-965eacadddbe admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:34.617697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049b6395-fc1b-4612-a9de-965eacadddbe admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:34.622983 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ead4d-5899-4f14-b82c-c584d7b6cb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:34.623604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ead4d-5899-4f14-b82c-c584d7b6cb7b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.624035 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ead4d-5899-4f14-b82c-c584d7b6cb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.624353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ead4d-5899-4f14-b82c-c584d7b6cb7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.625127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ead4d-5899-4f14-b82c-c584d7b6cb7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:34.626695 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ead4d-5899-4f14-b82c-c584d7b6cb7b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:34.642219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1356/2716] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:34.646524 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae45bec-8292-4e0c-a684-6dde9d6ab01b admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:34.656981 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae45bec-8292-4e0c-a684-6dde9d6ab01b admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:34.661975 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a2431-e87c-4e06-99b8-9b509177bb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:34.662592 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a2431-e87c-4e06-99b8-9b509177bb42 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.662871 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a2431-e87c-4e06-99b8-9b509177bb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.663146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a2431-e87c-4e06-99b8-9b509177bb42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.663936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a2431-e87c-4e06-99b8-9b509177bb42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:34.665491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a2431-e87c-4e06-99b8-9b509177bb42 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:34.677386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1361/2717] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:34.681495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ead4d-5899-4f14-b82c-c584d7b6cb7b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:34.690241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:34.694850 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ead4d-5899-4f14-b82c-c584d7b6cb7b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:34.704014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822963a0-e927-439d-b304-700e3f0f28c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:34.704945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822963a0-e927-439d-b304-700e3f0f28c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.705490 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822963a0-e927-439d-b304-700e3f0f28c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.706108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822963a0-e927-439d-b304-700e3f0f28c8 admin admin] PATH_INFO: `/v3/projects/846b7e3822a74b8195bea09184cffd0b/users/dc3d78a2653d41b296d650731d1e9815/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.723657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822963a0-e927-439d-b304-700e3f0f28c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846b7e3822a74b8195bea09184cffd0b, user_id=dc3d78a2653d41b296d650731d1e9815, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:34.727135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822963a0-e927-439d-b304-700e3f0f28c8 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:34.744610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1357/2718] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:34 2022] PUT /identity/v3/projects/846b7e3822a74b8195bea09184cffd0b/users/dc3d78a2653d41b296d650731d1e9815/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:34.755562 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250641f-8ce1-440f-b5f6-666cb7502dbc None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.756138 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250641f-8ce1-440f-b5f6-666cb7502dbc None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.756583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250641f-8ce1-440f-b5f6-666cb7502dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.787456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-e250641f-8ce1-440f-b5f6-666cb7502dbc None None] MFA Rules not processed for user `dc3d78a2653d41b296d650731d1e9815`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:34.788706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e250641f-8ce1-440f-b5f6-666cb7502dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:34.860199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1362/2719] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:34.872037 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822963a0-e927-439d-b304-700e3f0f28c8 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:34.873813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-822963a0-e927-439d-b304-700e3f0f28c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:34.939423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822963a0-e927-439d-b304-700e3f0f28c8 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:34.943297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01a815-cf2a-45bb-850f-6e811bbdf936 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:34.943962 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01a815-cf2a-45bb-850f-6e811bbdf936 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:34.944242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01a815-cf2a-45bb-850f-6e811bbdf936 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:34.944548 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01a815-cf2a-45bb-850f-6e811bbdf936 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:34.953602 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01a815-cf2a-45bb-850f-6e811bbdf936 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:34.955084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01a815-cf2a-45bb-850f-6e811bbdf936 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:34.964354 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1358/2720] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:34.984006 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250641f-8ce1-440f-b5f6-666cb7502dbc None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:35.035870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:35.053618 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250641f-8ce1-440f-b5f6-666cb7502dbc None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:35.059034 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:35.059710 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:35.059984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:35.060251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] PATH_INFO: `/v3/users/dc3d78a2653d41b296d650731d1e9815` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:35.061485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc3d78a2653d41b296d650731d1e9815)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:35.063407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:35.116283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc3d78a2653d41b296d650731d1e9815'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:35.116950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc3d78a2653d41b296d650731d1e9815'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:35.125578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc3d78a2653d41b296d650731d1e9815'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:35.137178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc3d78a2653d41b296d650731d1e9815'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:35.143775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc3d78a2653d41b296d650731d1e9815'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:35.149552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1363/2721] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:11:34 2022] DELETE /identity/v3/users/dc3d78a2653d41b296d650731d1e9815 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:11:35.154396 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e90fb-869c-4298-bd48-7c15d632d6b6 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:35.154683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e90fb-869c-4298-bd48-7c15d632d6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:35.154958 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e90fb-869c-4298-bd48-7c15d632d6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:35.174743 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-7e1e90fb-869c-4298-bd48-7c15d632d6b6 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:35.175989 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7e1e90fb-869c-4298-bd48-7c15d632d6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:35.240634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1359/2722] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:35.252498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:35.254744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:35.313355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3551a-d607-44b4-b75e-ab2e23808acb admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:35.317252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862792b-ad78-460d-a257-b1aa208b7032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:35.317875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862792b-ad78-460d-a257-b1aa208b7032 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:35.318147 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862792b-ad78-460d-a257-b1aa208b7032 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:35.318415 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862792b-ad78-460d-a257-b1aa208b7032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:35.329247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862792b-ad78-460d-a257-b1aa208b7032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:35.330558 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862792b-ad78-460d-a257-b1aa208b7032 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:35.340912 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1364/2723] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:35.705445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e90fb-869c-4298-bd48-7c15d632d6b6 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:35.707131 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7e1e90fb-869c-4298-bd48-7c15d632d6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:35.721044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e90fb-869c-4298-bd48-7c15d632d6b6 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:35.727060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:35.727847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:35.728129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:35.728410 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] PATH_INFO: `/v3/projects/846b7e3822a74b8195bea09184cffd0b` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:35.730215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=846b7e3822a74b8195bea09184cffd0b)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:35.732146 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:35.790704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846b7e3822a74b8195bea09184cffd0b', 'request_id': 'req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:35.797672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846b7e3822a74b8195bea09184cffd0b', 'request_id': 'req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:35.806869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846b7e3822a74b8195bea09184cffd0b', 'request_id': 'req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:35.817498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] The token cache is being invalidate because project 846b7e3822a74b8195bea09184cffd0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:11:35.817806 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:35.818423 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a74d9e77-3cf8-46f0-ab96-f79b3f8c3a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:35.819944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1360/2724] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:11:35 2022] DELETE /identity/v3/projects/846b7e3822a74b8195bea09184cffd0b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:35.825113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafe784-6384-42e0-8562-b2719001677a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:35.825393 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafe784-6384-42e0-8562-b2719001677a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:35.825661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bafe784-6384-42e0-8562-b2719001677a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:35.845272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-8bafe784-6384-42e0-8562-b2719001677a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:35.846483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8bafe784-6384-42e0-8562-b2719001677a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:35.908105 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1365/2725] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:35.916008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ae3b4-9ee3-46e0-bf21-8f7aba29048a None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:35.916327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ae3b4-9ee3-46e0-bf21-8f7aba29048a None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:35.916681 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ae3b4-9ee3-46e0-bf21-8f7aba29048a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:35.931473 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:35.937181 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-072ae3b4-9ee3-46e0-bf21-8f7aba29048a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:35.938924 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-072ae3b4-9ee3-46e0-bf21-8f7aba29048a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:35.947522 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1361/2726] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:35.952035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafe784-6384-42e0-8562-b2719001677a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:35.962013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bafe784-6384-42e0-8562-b2719001677a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:35.967866 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa75268-be02-4563-9cf4-298a7f189dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:35.968527 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa75268-be02-4563-9cf4-298a7f189dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:35.968807 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa75268-be02-4563-9cf4-298a7f189dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:35.969082 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa75268-be02-4563-9cf4-298a7f189dd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:35.969762 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa75268-be02-4563-9cf4-298a7f189dd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:11:35.970061 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa75268-be02-4563-9cf4-298a7f189dd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:35.971657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa75268-be02-4563-9cf4-298a7f189dd6 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:35.978815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:35.988228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1366/2727] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:11:35.989754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ae3b4-9ee3-46e0-bf21-8f7aba29048a None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:35.999587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ae3b4-9ee3-46e0-bf21-8f7aba29048a None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:36.005646 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ccf7e-312c-4991-9d30-0c4906eb16ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:36.006268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ccf7e-312c-4991-9d30-0c4906eb16ca admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:36.006546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ccf7e-312c-4991-9d30-0c4906eb16ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:36.006814 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ccf7e-312c-4991-9d30-0c4906eb16ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:36.007495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ccf7e-312c-4991-9d30-0c4906eb16ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:36.009455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ccf7e-312c-4991-9d30-0c4906eb16ca admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:36.040690 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1362/2728] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:11:35 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:11:36.045200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa75268-be02-4563-9cf4-298a7f189dd6 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:36.055925 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa75268-be02-4563-9cf4-298a7f189dd6 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:36.115951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bab4cf-b3a1-4a29-ae3d-f136dfaaa30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:36.116602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bab4cf-b3a1-4a29-ae3d-f136dfaaa30a admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:36.116885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bab4cf-b3a1-4a29-ae3d-f136dfaaa30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:36.117157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bab4cf-b3a1-4a29-ae3d-f136dfaaa30a admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:36.117846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bab4cf-b3a1-4a29-ae3d-f136dfaaa30a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:36.119799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bab4cf-b3a1-4a29-ae3d-f136dfaaa30a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:36.141481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1367/2729] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:11:36 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:36.145866 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ccf7e-312c-4991-9d30-0c4906eb16ca admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:36.156917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ccf7e-312c-4991-9d30-0c4906eb16ca admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:36.162743 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbafde-8892-453c-984f-9c7f4b4a5695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:36.163324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbafde-8892-453c-984f-9c7f4b4a5695 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:36.163600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbafde-8892-453c-984f-9c7f4b4a5695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:36.163869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbafde-8892-453c-984f-9c7f4b4a5695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:36.164607 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbafde-8892-453c-984f-9c7f4b4a5695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:36.166245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbafde-8892-453c-984f-9c7f4b4a5695 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:36.176707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1363/2730] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:11:36.180984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bab4cf-b3a1-4a29-ae3d-f136dfaaa30a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:36.191469 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bab4cf-b3a1-4a29-ae3d-f136dfaaa30a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:36.196368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b4944-1cb6-4ee4-aa7f-7bdb3fc7cff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:36.196976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4944-1cb6-4ee4-aa7f-7bdb3fc7cff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:36.197251 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4944-1cb6-4ee4-aa7f-7bdb3fc7cff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:36.197519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4944-1cb6-4ee4-aa7f-7bdb3fc7cff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:36.198347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b4944-1cb6-4ee4-aa7f-7bdb3fc7cff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:36.199953 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b4944-1cb6-4ee4-aa7f-7bdb3fc7cff4 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:36.210542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1368/2731] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:36.215371 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbafde-8892-453c-984f-9c7f4b4a5695 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:36.229731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbafde-8892-453c-984f-9c7f4b4a5695 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:36.235380 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d089541-5a3d-4aaf-a7d6-7478e56ee1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:36.236058 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d089541-5a3d-4aaf-a7d6-7478e56ee1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:36.236342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d089541-5a3d-4aaf-a7d6-7478e56ee1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:36.236625 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d089541-5a3d-4aaf-a7d6-7478e56ee1e6 admin admin] PATH_INFO: `/v3/projects/f394f5b13a6645baa7c20c1faf0ca554/users/bf3d0b8b6e0a466185e39b324686fad9/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:36.255502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d089541-5a3d-4aaf-a7d6-7478e56ee1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f394f5b13a6645baa7c20c1faf0ca554, user_id=bf3d0b8b6e0a466185e39b324686fad9, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:36.257680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d089541-5a3d-4aaf-a7d6-7478e56ee1e6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:36.278980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1364/2732] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:36 2022] PUT /identity/v3/projects/f394f5b13a6645baa7c20c1faf0ca554/users/bf3d0b8b6e0a466185e39b324686fad9/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:36.291200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2db06-4e25-4451-912d-8286e32b731c None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:36.291518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2db06-4e25-4451-912d-8286e32b731c None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:36.291840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2db06-4e25-4451-912d-8286e32b731c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:36.326330 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-85f2db06-4e25-4451-912d-8286e32b731c None None] MFA Rules not processed for user `bf3d0b8b6e0a466185e39b324686fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:36.327573 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-85f2db06-4e25-4451-912d-8286e32b731c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:36.409754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1369/2733] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:36.421429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d089541-5a3d-4aaf-a7d6-7478e56ee1e6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:36.423358 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6d089541-5a3d-4aaf-a7d6-7478e56ee1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:36.483964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d089541-5a3d-4aaf-a7d6-7478e56ee1e6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:36.487909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8e48d-fdce-4f1e-ae26-fcc2dc51428a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:36.488921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e48d-fdce-4f1e-ae26-fcc2dc51428a service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:36.489199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e48d-fdce-4f1e-ae26-fcc2dc51428a service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:36.489465 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8e48d-fdce-4f1e-ae26-fcc2dc51428a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:36.491245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabaf83-860e-49ba-ad6e-31d240405088 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:36.491473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabaf83-860e-49ba-ad6e-31d240405088 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:36.491744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabaf83-860e-49ba-ad6e-31d240405088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:36.498226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8e48d-fdce-4f1e-ae26-fcc2dc51428a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:36.499661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce8e48d-fdce-4f1e-ae26-fcc2dc51428a service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:36.508627 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1365/2734] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:36.511206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7dabaf83-860e-49ba-ad6e-31d240405088 None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:36.512432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7dabaf83-860e-49ba-ad6e-31d240405088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:36.570994 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1370/2735] 10.222.0.49 () {58 vars in 967 bytes} [Mon Aug 1 12:11:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:11:36.585095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8e48d-fdce-4f1e-ae26-fcc2dc51428a service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:36.586510 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6ce8e48d-fdce-4f1e-ae26-fcc2dc51428a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:36.662608 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8e48d-fdce-4f1e-ae26-fcc2dc51428a service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:36.669151 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da3762-c525-4477-9ae9-35739bcbff78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:36.669933 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3762-c525-4477-9ae9-35739bcbff78 service swift] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:36.670056 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3762-c525-4477-9ae9-35739bcbff78 service swift] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:36.670327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3762-c525-4477-9ae9-35739bcbff78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:36.680547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da3762-c525-4477-9ae9-35739bcbff78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:36.682186 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da3762-c525-4477-9ae9-35739bcbff78 service swift] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:36.691502 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1366/2736] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:11:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:11:39.117061 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabaf83-860e-49ba-ad6e-31d240405088 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:39.131325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:39.187208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabaf83-860e-49ba-ad6e-31d240405088 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:39.192369 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:39.193025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:39.193320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:39.193579 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] PATH_INFO: `/v3/users/bf3d0b8b6e0a466185e39b324686fad9` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:39.194849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf3d0b8b6e0a466185e39b324686fad9)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:39.196820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:39.253199 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf3d0b8b6e0a466185e39b324686fad9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:39.253882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf3d0b8b6e0a466185e39b324686fad9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:39.262252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf3d0b8b6e0a466185e39b324686fad9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:39.274141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf3d0b8b6e0a466185e39b324686fad9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:39.281125 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf3d0b8b6e0a466185e39b324686fad9'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:39.287740 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1371/2737] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:11:39 2022] DELETE /identity/v3/users/bf3d0b8b6e0a466185e39b324686fad9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:39.292573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383fbaa-867a-425e-a75b-ba6c3b9839d2 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:39.293063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383fbaa-867a-425e-a75b-ba6c3b9839d2 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:39.293162 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383fbaa-867a-425e-a75b-ba6c3b9839d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:39.312859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b383fbaa-867a-425e-a75b-ba6c3b9839d2 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:39.314079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b383fbaa-867a-425e-a75b-ba6c3b9839d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:39.344872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:39.378636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1367/2738] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:11:39.388682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:39.390631 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:39.453770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ccd20a-d6b7-4567-b726-be251392bdb4 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:39.457868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c956c4-fb1a-41e2-a647-85830836168f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:39.458490 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c956c4-fb1a-41e2-a647-85830836168f service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:39.458761 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c956c4-fb1a-41e2-a647-85830836168f service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:39.459034 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c956c4-fb1a-41e2-a647-85830836168f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:39.468787 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c956c4-fb1a-41e2-a647-85830836168f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:39.470215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c956c4-fb1a-41e2-a647-85830836168f service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:39.479822 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1372/2739] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:39.769154 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383fbaa-867a-425e-a75b-ba6c3b9839d2 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:39.770828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b383fbaa-867a-425e-a75b-ba6c3b9839d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:39.785725 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383fbaa-867a-425e-a75b-ba6c3b9839d2 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:39.790951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:39.791660 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:39.792030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:39.792193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] PATH_INFO: `/v3/projects/f394f5b13a6645baa7c20c1faf0ca554` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:39.793583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f394f5b13a6645baa7c20c1faf0ca554)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:39.795688 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:39.846956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f394f5b13a6645baa7c20c1faf0ca554', 'request_id': 'req-1836e93d-ece9-4f3a-8966-79b4bd686117'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:39.853680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f394f5b13a6645baa7c20c1faf0ca554', 'request_id': 'req-1836e93d-ece9-4f3a-8966-79b4bd686117'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:39.861783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f394f5b13a6645baa7c20c1faf0ca554', 'request_id': 'req-1836e93d-ece9-4f3a-8966-79b4bd686117'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:39.872998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] The token cache is being invalidate because project f394f5b13a6645baa7c20c1faf0ca554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:11:39.873283 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:39.873929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-1836e93d-ece9-4f3a-8966-79b4bd686117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:11:39.875477 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1368/2740] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:11:39 2022] DELETE /identity/v3/projects/f394f5b13a6645baa7c20c1faf0ca554 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:39.882285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae348dc-6dba-4239-ad5f-59e3074a049d None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:39.882508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae348dc-6dba-4239-ad5f-59e3074a049d None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:39.882786 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae348dc-6dba-4239-ad5f-59e3074a049d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:39.902547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6ae348dc-6dba-4239-ad5f-59e3074a049d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:39.903936 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6ae348dc-6dba-4239-ad5f-59e3074a049d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:39.949011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:39.999601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1373/2741] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:40.007947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbbba5-9e65-457a-a034-a1adef10d7bf None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:40.008245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbbba5-9e65-457a-a034-a1adef10d7bf None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:40.008550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbbba5-9e65-457a-a034-a1adef10d7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:40.028992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-08bbbba5-9e65-457a-a034-a1adef10d7bf None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:40.030220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-08bbbba5-9e65-457a-a034-a1adef10d7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:40.039886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1369/2742] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:40.044512 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae348dc-6dba-4239-ad5f-59e3074a049d None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:40.054917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae348dc-6dba-4239-ad5f-59e3074a049d None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:40.060046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8216f30-9f85-4324-9aa1-45e72e73abe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:40.060687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8216f30-9f85-4324-9aa1-45e72e73abe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:40.060963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8216f30-9f85-4324-9aa1-45e72e73abe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:40.061231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8216f30-9f85-4324-9aa1-45e72e73abe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:40.061930 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8216f30-9f85-4324-9aa1-45e72e73abe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:11:40.062229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8216f30-9f85-4324-9aa1-45e72e73abe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:40.063823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8216f30-9f85-4324-9aa1-45e72e73abe5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:40.076659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1374/2743] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:11:40.081805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbbba5-9e65-457a-a034-a1adef10d7bf None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:40.094289 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbbba5-9e65-457a-a034-a1adef10d7bf None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:40.101492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da310c-4833-49ce-822f-bb9f39eecca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:40.102374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da310c-4833-49ce-822f-bb9f39eecca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:40.102747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da310c-4833-49ce-822f-bb9f39eecca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:40.103025 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da310c-4833-49ce-822f-bb9f39eecca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:40.104148 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da310c-4833-49ce-822f-bb9f39eecca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:40.106137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da310c-4833-49ce-822f-bb9f39eecca7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:40.137531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1370/2744] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:11:40 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:11:40.143452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8216f30-9f85-4324-9aa1-45e72e73abe5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:40.159250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8216f30-9f85-4324-9aa1-45e72e73abe5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:40.235670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553aba3b-7680-45a2-ae64-12344a6d2823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:40.236346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553aba3b-7680-45a2-ae64-12344a6d2823 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:40.236695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553aba3b-7680-45a2-ae64-12344a6d2823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:40.236890 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553aba3b-7680-45a2-ae64-12344a6d2823 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:40.237583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553aba3b-7680-45a2-ae64-12344a6d2823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:40.239874 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553aba3b-7680-45a2-ae64-12344a6d2823 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:40.268768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1375/2745] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:11:40 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:40.273626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da310c-4833-49ce-822f-bb9f39eecca7 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:40.286497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da310c-4833-49ce-822f-bb9f39eecca7 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:40.293590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45298d13-4b54-44b9-bd3f-6575a1c2a020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:40.294253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45298d13-4b54-44b9-bd3f-6575a1c2a020 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:40.294571 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45298d13-4b54-44b9-bd3f-6575a1c2a020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:40.294909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45298d13-4b54-44b9-bd3f-6575a1c2a020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:40.295761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45298d13-4b54-44b9-bd3f-6575a1c2a020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:40.298326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45298d13-4b54-44b9-bd3f-6575a1c2a020 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:40.314485 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1371/2746] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:40.317335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553aba3b-7680-45a2-ae64-12344a6d2823 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:40.328093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553aba3b-7680-45a2-ae64-12344a6d2823 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:40.333438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba2599-8477-44ee-a53d-d2917ef265e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:40.334084 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba2599-8477-44ee-a53d-d2917ef265e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:40.334361 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba2599-8477-44ee-a53d-d2917ef265e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:40.334633 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba2599-8477-44ee-a53d-d2917ef265e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:40.335335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba2599-8477-44ee-a53d-d2917ef265e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:40.337093 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba2599-8477-44ee-a53d-d2917ef265e5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:40.349462 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1376/2747] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:40.356011 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45298d13-4b54-44b9-bd3f-6575a1c2a020 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:40.371214 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45298d13-4b54-44b9-bd3f-6575a1c2a020 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:40.376928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec66e6-0cc0-43ba-b2be-36adc97f0535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:40.377600 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec66e6-0cc0-43ba-b2be-36adc97f0535 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:40.377875 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec66e6-0cc0-43ba-b2be-36adc97f0535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:40.378242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec66e6-0cc0-43ba-b2be-36adc97f0535 admin admin] PATH_INFO: `/v3/projects/d21aa81cdb8f40f5931615c0d85895a2/users/11c07e105d1e420faa1b4b2be7dbe09c/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:40.394028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec66e6-0cc0-43ba-b2be-36adc97f0535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d21aa81cdb8f40f5931615c0d85895a2, user_id=11c07e105d1e420faa1b4b2be7dbe09c, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:40.396701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec66e6-0cc0-43ba-b2be-36adc97f0535 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:40.411916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1372/2748] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:40 2022] PUT /identity/v3/projects/d21aa81cdb8f40f5931615c0d85895a2/users/11c07e105d1e420faa1b4b2be7dbe09c/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:40.417214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd960-ce72-4390-8eda-861d1231ec7a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:40.417496 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd960-ce72-4390-8eda-861d1231ec7a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:40.417775 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cd960-ce72-4390-8eda-861d1231ec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:40.439156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-544cd960-ce72-4390-8eda-861d1231ec7a None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:40.440511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-544cd960-ce72-4390-8eda-861d1231ec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:40.506237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1377/2749] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:40.516534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec66e6-0cc0-43ba-b2be-36adc97f0535 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:40.518744 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-feec66e6-0cc0-43ba-b2be-36adc97f0535 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:40.583923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec66e6-0cc0-43ba-b2be-36adc97f0535 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:40.588487 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a82a41-5067-4f23-a09d-6dc732e6341b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:40.589149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a82a41-5067-4f23-a09d-6dc732e6341b service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:40.589468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a82a41-5067-4f23-a09d-6dc732e6341b service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:40.589776 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a82a41-5067-4f23-a09d-6dc732e6341b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:40.601198 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a82a41-5067-4f23-a09d-6dc732e6341b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:40.602687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a82a41-5067-4f23-a09d-6dc732e6341b service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:40.607167 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:40.617135 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1373/2750] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:11:44.738954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352f02-3dc4-4bd9-9315-35ceccaf8f4a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:44.739513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352f02-3dc4-4bd9-9315-35ceccaf8f4a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:44.739513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352f02-3dc4-4bd9-9315-35ceccaf8f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:44.776928 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9f352f02-3dc4-4bd9-9315-35ceccaf8f4a None None] MFA Rules not processed for user `11c07e105d1e420faa1b4b2be7dbe09c`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:44.778381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9f352f02-3dc4-4bd9-9315-35ceccaf8f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:44.815245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:44.867670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1378/2751] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:44.872649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a82a41-5067-4f23-a09d-6dc732e6341b service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:44.883482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a82a41-5067-4f23-a09d-6dc732e6341b service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:44.888629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e6c35-a053-4f95-b496-81df753a6f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:44.889413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e6c35-a053-4f95-b496-81df753a6f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:44.889697 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e6c35-a053-4f95-b496-81df753a6f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:44.889976 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e6c35-a053-4f95-b496-81df753a6f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:44.890682 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e6c35-a053-4f95-b496-81df753a6f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:44.892712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e6c35-a053-4f95-b496-81df753a6f81 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:44.933337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1374/2752] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:11:44 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:11:44.935429 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f352f02-3dc4-4bd9-9315-35ceccaf8f4a None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:44.949040 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f352f02-3dc4-4bd9-9315-35ceccaf8f4a None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:45.024528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5dad4-4445-4bfd-afbf-724b3793a8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:45.025237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dad4-4445-4bfd-afbf-724b3793a8b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:45.025724 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dad4-4445-4bfd-afbf-724b3793a8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:45.026145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5dad4-4445-4bfd-afbf-724b3793a8b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:45.026703 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5dad4-4445-4bfd-afbf-724b3793a8b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:45.028768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5dad4-4445-4bfd-afbf-724b3793a8b2 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:45.060510 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1379/2753] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:11:44 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:11:45.064963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e6c35-a053-4f95-b496-81df753a6f81 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:45.077623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e6c35-a053-4f95-b496-81df753a6f81 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:45.083482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e2d08-1318-492b-b4c5-9ce06a73d70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:45.084480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e2d08-1318-492b-b4c5-9ce06a73d70b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:45.084889 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e2d08-1318-492b-b4c5-9ce06a73d70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:45.085224 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e2d08-1318-492b-b4c5-9ce06a73d70b admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:45.085991 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e2d08-1318-492b-b4c5-9ce06a73d70b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:45.087721 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e2d08-1318-492b-b4c5-9ce06a73d70b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:45.100720 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1375/2754] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:45.106321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5dad4-4445-4bfd-afbf-724b3793a8b2 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:45.118680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5dad4-4445-4bfd-afbf-724b3793a8b2 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:45.124615 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f19bd-955f-4517-af85-da9ff996716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:45.125476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f19bd-955f-4517-af85-da9ff996716a admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:45.125804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f19bd-955f-4517-af85-da9ff996716a admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:45.126156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f19bd-955f-4517-af85-da9ff996716a admin admin] PATH_INFO: `/v3/projects/dac130aac76044aba7fbfec95a9e9d9f/users/51474b8d57594305ac4ead352a29e7e5/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:45.143603 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f19bd-955f-4517-af85-da9ff996716a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac130aac76044aba7fbfec95a9e9d9f, user_id=51474b8d57594305ac4ead352a29e7e5, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:45.146317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f19bd-955f-4517-af85-da9ff996716a admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:45.165186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1380/2755] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:45 2022] PUT /identity/v3/projects/dac130aac76044aba7fbfec95a9e9d9f/users/51474b8d57594305ac4ead352a29e7e5/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:45.170303 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e2d08-1318-492b-b4c5-9ce06a73d70b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:45.185179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e2d08-1318-492b-b4c5-9ce06a73d70b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:45.254038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078c833-ff8c-4a7d-a711-b76c2d34c401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:45.254583 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c833-ff8c-4a7d-a711-b76c2d34c401 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:45.254867 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c833-ff8c-4a7d-a711-b76c2d34c401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:45.255125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078c833-ff8c-4a7d-a711-b76c2d34c401 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:45.255833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078c833-ff8c-4a7d-a711-b76c2d34c401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:45.257500 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078c833-ff8c-4a7d-a711-b76c2d34c401 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:45.269726 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1376/2756] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:11:45.274091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f19bd-955f-4517-af85-da9ff996716a admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:45.285749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f19bd-955f-4517-af85-da9ff996716a admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:45.291663 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe3315-14e6-4115-8f0f-1b6270ac75b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:45.292394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe3315-14e6-4115-8f0f-1b6270ac75b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:45.292673 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe3315-14e6-4115-8f0f-1b6270ac75b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:45.292939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbe3315-14e6-4115-8f0f-1b6270ac75b8 admin admin] PATH_INFO: `/v3/projects/dac130aac76044aba7fbfec95a9e9d9f/users/51474b8d57594305ac4ead352a29e7e5/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:45.295076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe3315-14e6-4115-8f0f-1b6270ac75b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac130aac76044aba7fbfec95a9e9d9f, user_id=51474b8d57594305ac4ead352a29e7e5, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:45.297772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbe3315-14e6-4115-8f0f-1b6270ac75b8 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:45.309920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1381/2757] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:11:45 2022] PUT /identity/v3/projects/dac130aac76044aba7fbfec95a9e9d9f/users/51474b8d57594305ac4ead352a29e7e5/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:11:50.922114 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a72290-cbb8-4e64-abea-8e1f6ac44e06 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:50.922672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a72290-cbb8-4e64-abea-8e1f6ac44e06 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:50.922672 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a72290-cbb8-4e64-abea-8e1f6ac44e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:50.955451 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-44a72290-cbb8-4e64-abea-8e1f6ac44e06 None None] MFA Rules not processed for user `51474b8d57594305ac4ead352a29e7e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:11:50.956759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-44a72290-cbb8-4e64-abea-8e1f6ac44e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:51.042941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1377/2758] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:11:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:11:51.059242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe3315-14e6-4115-8f0f-1b6270ac75b8 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:11:51.062497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-bfbe3315-14e6-4115-8f0f-1b6270ac75b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:11:51.157883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbe3315-14e6-4115-8f0f-1b6270ac75b8 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:11:51.163767 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8b1be-9164-44d1-a363-1f36b21385b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:11:51.164455 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b1be-9164-44d1-a363-1f36b21385b9 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:11:51.164742 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b1be-9164-44d1-a363-1f36b21385b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:11:51.165017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8b1be-9164-44d1-a363-1f36b21385b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:11:51.187670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:11:51.249439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8b1be-9164-44d1-a363-1f36b21385b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:11:51.251542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8b1be-9164-44d1-a363-1f36b21385b9 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:11:51.263355 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1382/2759] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:11:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:12:01.039523 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=78,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:12:01.258228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=19 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:12:08.870023 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4803-c9a7-4a0d-afd1-8e957a2785ce None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:08.870614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4803-c9a7-4a0d-afd1-8e957a2785ce None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:08.870614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4803-c9a7-4a0d-afd1-8e957a2785ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:08.891046 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-22eb4803-c9a7-4a0d-afd1-8e957a2785ce None None] MFA Rules not processed for user `11c07e105d1e420faa1b4b2be7dbe09c`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:12:08.892418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-22eb4803-c9a7-4a0d-afd1-8e957a2785ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:08.900835 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1378/2760] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:12:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:12:16.673513 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8b1be-9164-44d1-a363-1f36b21385b9 service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:16.675641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ecf8b1be-9164-44d1-a363-1f36b21385b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:16.766329 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8b1be-9164-44d1-a363-1f36b21385b9 service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:16.770225 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fb625-6717-4818-aa3c-61a28a26458c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:16.770879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fb625-6717-4818-aa3c-61a28a26458c service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:16.771162 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fb625-6717-4818-aa3c-61a28a26458c service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:16.771435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fb625-6717-4818-aa3c-61a28a26458c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:16.773608 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-462fb625-6717-4818-aa3c-61a28a26458c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:16.876571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fb625-6717-4818-aa3c-61a28a26458c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:16.878665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fb625-6717-4818-aa3c-61a28a26458c service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:16.898417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1383/2761] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:12:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:12:17.885960 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb4803-c9a7-4a0d-afd1-8e957a2785ce None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:17.902719 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb4803-c9a7-4a0d-afd1-8e957a2785ce None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:17.909092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350cb1c-fb2b-4849-aa71-21e583357723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:17.910086 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350cb1c-fb2b-4849-aa71-21e583357723 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:17.910509 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350cb1c-fb2b-4849-aa71-21e583357723 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:17.910942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350cb1c-fb2b-4849-aa71-21e583357723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:17.914108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c350cb1c-fb2b-4849-aa71-21e583357723 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:17.928272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350cb1c-fb2b-4849-aa71-21e583357723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:17.930072 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c350cb1c-fb2b-4849-aa71-21e583357723 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:17.945187 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1379/2762] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:12:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:12:19.519044 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fb625-6717-4818-aa3c-61a28a26458c service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:19.528682 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fb625-6717-4818-aa3c-61a28a26458c service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:19.537343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b15572-d6ae-4ea2-adf4-faaad3a795e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:19.538059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b15572-d6ae-4ea2-adf4-faaad3a795e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:19.538409 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b15572-d6ae-4ea2-adf4-faaad3a795e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:19.538726 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b15572-d6ae-4ea2-adf4-faaad3a795e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:19.540878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e3b15572-d6ae-4ea2-adf4-faaad3a795e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:19.629417 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b15572-d6ae-4ea2-adf4-faaad3a795e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:19.630885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b15572-d6ae-4ea2-adf4-faaad3a795e2 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:19.642493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1384/2763] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:12:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:12:21.908121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350cb1c-fb2b-4849-aa71-21e583357723 service nova] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:21.924128 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c350cb1c-fb2b-4849-aa71-21e583357723 service nova] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:21.928418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2b4ed-49c8-439e-863c-a090098e6257 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:21.929149 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2b4ed-49c8-439e-863c-a090098e6257 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:21.929536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2b4ed-49c8-439e-863c-a090098e6257 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:21.929916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2b4ed-49c8-439e-863c-a090098e6257 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:21.931952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b4b2b4ed-49c8-439e-863c-a090098e6257 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:21.945926 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2b4ed-49c8-439e-863c-a090098e6257 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:21.947767 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2b4ed-49c8-439e-863c-a090098e6257 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:21.958858 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1380/2764] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:12:23.552333 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b15572-d6ae-4ea2-adf4-faaad3a795e2 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:23.554535 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e3b15572-d6ae-4ea2-adf4-faaad3a795e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:23.645691 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2b4ed-49c8-439e-863c-a090098e6257 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:23.648388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b4b2b4ed-49c8-439e-863c-a090098e6257 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:23.674489 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2b4ed-49c8-439e-863c-a090098e6257 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:23.685389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:12:23.689504 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:23.690712 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:23.691093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:23.691464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] PATH_INFO: `/v3/users/e98a16b117294e10b9d0c8c721d8e114` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:23.696028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e98a16b117294e10b9d0c8c721d8e114)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:23.701432 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:23.724144 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b15572-d6ae-4ea2-adf4-faaad3a795e2 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:23.744929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f0738-5474-43bc-8f2f-52f1d2837b81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:23.745604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f0738-5474-43bc-8f2f-52f1d2837b81 service placement] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:23.746077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f0738-5474-43bc-8f2f-52f1d2837b81 service placement] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:23.746528 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f0738-5474-43bc-8f2f-52f1d2837b81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:23.770931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c84f0738-5474-43bc-8f2f-52f1d2837b81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:23.794091 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f0738-5474-43bc-8f2f-52f1d2837b81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:23.795942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f0738-5474-43bc-8f2f-52f1d2837b81 service placement] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:23.819794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1385/2765] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:12:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:12:23.840827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e98a16b117294e10b9d0c8c721d8e114'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:23.877483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e98a16b117294e10b9d0c8c721d8e114'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:23.898448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e98a16b117294e10b9d0c8c721d8e114'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:23.900654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e98a16b117294e10b9d0c8c721d8e114'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:23.902973 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-615ff280-0de8-41f8-9215-a4e0d4debd92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e98a16b117294e10b9d0c8c721d8e114'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:23.957638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1381/2766] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:12:23 2022] DELETE /identity/v3/users/e98a16b117294e10b9d0c8c721d8e114 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:12:24.186570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f0738-5474-43bc-8f2f-52f1d2837b81 service placement] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:24.198917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f0738-5474-43bc-8f2f-52f1d2837b81 service placement] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:24.274508 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:24.275174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:24.275459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:24.275737 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] PATH_INFO: `/v3/projects/1971a84abd0145358138a0ef1b2501a9` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:24.277136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1971a84abd0145358138a0ef1b2501a9)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:24.279136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:24.337816 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1971a84abd0145358138a0ef1b2501a9', 'request_id': 'req-70a5ceec-1d97-4c57-83db-0c04d0ace40b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:24.346156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1971a84abd0145358138a0ef1b2501a9', 'request_id': 'req-70a5ceec-1d97-4c57-83db-0c04d0ace40b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:24.356016 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1971a84abd0145358138a0ef1b2501a9', 'request_id': 'req-70a5ceec-1d97-4c57-83db-0c04d0ace40b'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:24.378025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] The token cache is being invalidate because project 1971a84abd0145358138a0ef1b2501a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:12:24.378025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:24.379008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-70a5ceec-1d97-4c57-83db-0c04d0ace40b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:24.381021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1386/2767] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:12:24 2022] DELETE /identity/v3/projects/1971a84abd0145358138a0ef1b2501a9 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:12:24.386505 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd783643-b651-4698-bf89-ee614d7acb94 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:24.386909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd783643-b651-4698-bf89-ee614d7acb94 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:24.387192 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd783643-b651-4698-bf89-ee614d7acb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:24.409514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-fd783643-b651-4698-bf89-ee614d7acb94 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:12:24.411238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-fd783643-b651-4698-bf89-ee614d7acb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:24.542025 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1382/2768] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:12:24.544877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6388153-0d05-4890-bea8-43713ece5ee4 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:24.545250 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6388153-0d05-4890-bea8-43713ece5ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:24.545748 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6388153-0d05-4890-bea8-43713ece5ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:24.581130 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-f6388153-0d05-4890-bea8-43713ece5ee4 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:12:24.582800 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f6388153-0d05-4890-bea8-43713ece5ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:24.595373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1387/2769] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:12:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:12:24.599466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd783643-b651-4698-bf89-ee614d7acb94 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:24.611854 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd783643-b651-4698-bf89-ee614d7acb94 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:24.620574 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bd668-19b6-441f-93c2-e551682dc3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:24.621271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd668-19b6-441f-93c2-e551682dc3a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:24.621793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd668-19b6-441f-93c2-e551682dc3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:24.622513 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406bd668-19b6-441f-93c2-e551682dc3a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:24.623709 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bd668-19b6-441f-93c2-e551682dc3a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:12:24.624347 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bd668-19b6-441f-93c2-e551682dc3a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:24.627090 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406bd668-19b6-441f-93c2-e551682dc3a2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:24.647384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1383/2770] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:12:24.652693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6388153-0d05-4890-bea8-43713ece5ee4 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:24.662928 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6388153-0d05-4890-bea8-43713ece5ee4 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:24.668408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d46330-c1f8-4344-adfd-79fc76a266e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:24.669543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d46330-c1f8-4344-adfd-79fc76a266e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:24.670041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d46330-c1f8-4344-adfd-79fc76a266e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:24.670380 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d46330-c1f8-4344-adfd-79fc76a266e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:24.671153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d46330-c1f8-4344-adfd-79fc76a266e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:24.673253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d46330-c1f8-4344-adfd-79fc76a266e9 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:24.709831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1388/2771] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:12:24 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:12:24.717413 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bd668-19b6-441f-93c2-e551682dc3a2 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:24.732084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406bd668-19b6-441f-93c2-e551682dc3a2 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:24.767488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:12:24.820455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866185b8-e907-4648-953f-4ef07eb0ff49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:24.821629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866185b8-e907-4648-953f-4ef07eb0ff49 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:24.822103 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866185b8-e907-4648-953f-4ef07eb0ff49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:24.822567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866185b8-e907-4648-953f-4ef07eb0ff49 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:24.823827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866185b8-e907-4648-953f-4ef07eb0ff49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:24.827642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866185b8-e907-4648-953f-4ef07eb0ff49 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:24.878998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1384/2772] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:12:24 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:12:24.883886 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d46330-c1f8-4344-adfd-79fc76a266e9 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:24.897962 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d46330-c1f8-4344-adfd-79fc76a266e9 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:24.903676 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c62cd9-f69f-4fce-a02d-b2932171fb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:24.904718 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c62cd9-f69f-4fce-a02d-b2932171fb76 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:24.905104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c62cd9-f69f-4fce-a02d-b2932171fb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:24.905472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c62cd9-f69f-4fce-a02d-b2932171fb76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:24.906291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c62cd9-f69f-4fce-a02d-b2932171fb76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:24.908374 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c62cd9-f69f-4fce-a02d-b2932171fb76 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:24.925916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1389/2773] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:12:24.927218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866185b8-e907-4648-953f-4ef07eb0ff49 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:24.939387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866185b8-e907-4648-953f-4ef07eb0ff49 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:24.948016 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df900923-2a51-4213-995b-517f0c352fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:24.948784 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df900923-2a51-4213-995b-517f0c352fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:24.949060 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df900923-2a51-4213-995b-517f0c352fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:24.949538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df900923-2a51-4213-995b-517f0c352fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:24.950312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df900923-2a51-4213-995b-517f0c352fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:24.952238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df900923-2a51-4213-995b-517f0c352fd6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:24.969591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1385/2774] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:12:24.980028 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c62cd9-f69f-4fce-a02d-b2932171fb76 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:25.013336 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c62cd9-f69f-4fce-a02d-b2932171fb76 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:25.024768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde90b8e-73c5-46a8-a708-fe7ed4ff2767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:25.027310 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90b8e-73c5-46a8-a708-fe7ed4ff2767 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:25.027973 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90b8e-73c5-46a8-a708-fe7ed4ff2767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:25.028588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90b8e-73c5-46a8-a708-fe7ed4ff2767 admin admin] PATH_INFO: `/v3/projects/ac2e8512439547d9affb144ee0df1c39/users/d19967d791b94e0eac324065e8e2f232/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:25.066424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde90b8e-73c5-46a8-a708-fe7ed4ff2767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2e8512439547d9affb144ee0df1c39, user_id=d19967d791b94e0eac324065e8e2f232, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:25.075059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde90b8e-73c5-46a8-a708-fe7ed4ff2767 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:25.116754 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1390/2775] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:12:24 2022] PUT /identity/v3/projects/ac2e8512439547d9affb144ee0df1c39/users/d19967d791b94e0eac324065e8e2f232/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:12:25.119568 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710d415-d8ee-47ec-b06d-0ecb5093c4e8 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:25.119980 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710d415-d8ee-47ec-b06d-0ecb5093c4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:25.120407 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b710d415-d8ee-47ec-b06d-0ecb5093c4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:25.160978 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-b710d415-d8ee-47ec-b06d-0ecb5093c4e8 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:12:25.162987 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b710d415-d8ee-47ec-b06d-0ecb5093c4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:25.336773 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1386/2776] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:12:25.352624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde90b8e-73c5-46a8-a708-fe7ed4ff2767 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:25.359976 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-cde90b8e-73c5-46a8-a708-fe7ed4ff2767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:25.373314 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:12:25.512446 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde90b8e-73c5-46a8-a708-fe7ed4ff2767 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:25.519107 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238941b-78f1-43ee-8404-d735004a45ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:25.520714 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238941b-78f1-43ee-8404-d735004a45ca service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:25.520926 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238941b-78f1-43ee-8404-d735004a45ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:25.521254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238941b-78f1-43ee-8404-d735004a45ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:25.534730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238941b-78f1-43ee-8404-d735004a45ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:25.536231 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238941b-78f1-43ee-8404-d735004a45ca service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:25.561052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1391/2777] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:12:28.607984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710d415-d8ee-47ec-b06d-0ecb5093c4e8 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:28.611268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b710d415-d8ee-47ec-b06d-0ecb5093c4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:28.645753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b710d415-d8ee-47ec-b06d-0ecb5093c4e8 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:28.654576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e426e-8b0a-4993-a88f-2a59e99879a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:28.656319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e426e-8b0a-4993-a88f-2a59e99879a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:28.656319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e426e-8b0a-4993-a88f-2a59e99879a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:28.656914 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e426e-8b0a-4993-a88f-2a59e99879a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:28.660189 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-473e426e-8b0a-4993-a88f-2a59e99879a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:28.838743 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e426e-8b0a-4993-a88f-2a59e99879a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:28.841218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e426e-8b0a-4993-a88f-2a59e99879a2 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:28.863758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1387/2778] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:12:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:12:32.971107 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238941b-78f1-43ee-8404-d735004a45ca service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:32.974276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d238941b-78f1-43ee-8404-d735004a45ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:33.000145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238941b-78f1-43ee-8404-d735004a45ca service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:33.005392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:33.006022 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:33.007166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:33.007166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] PATH_INFO: `/v3/users/24f34c5beaad416188a32cf0860dd93e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:33.008397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f34c5beaad416188a32cf0860dd93e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:33.011104 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:33.079094 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f34c5beaad416188a32cf0860dd93e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:33.080353 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f34c5beaad416188a32cf0860dd93e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:33.095759 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f34c5beaad416188a32cf0860dd93e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:33.121401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ff71c-85ad-4a9c-8291-c95011715502 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:33.122293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ff71c-85ad-4a9c-8291-c95011715502 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:33.122293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ff71c-85ad-4a9c-8291-c95011715502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:33.130560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f34c5beaad416188a32cf0860dd93e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:33.158394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f34c5beaad416188a32cf0860dd93e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:33.170733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1392/2779] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:12:32 2022] DELETE /identity/v3/users/24f34c5beaad416188a32cf0860dd93e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:12:33.191305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-e98ff71c-85ad-4a9c-8291-c95011715502 None None] MFA Rules not processed for user `d19967d791b94e0eac324065e8e2f232`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:12:33.193026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e98ff71c-85ad-4a9c-8291-c95011715502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:33.264788 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:12:33.344615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1388/2780] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:12:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:12:33.359265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:33.362755 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:33.495785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659e3b91-f6a3-4352-b4ab-0a45a842a651 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:33.503391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ff71c-85ad-4a9c-8291-c95011715502 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:33.504801 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e98ff71c-85ad-4a9c-8291-c95011715502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:33.506987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d401fa2-c553-4f47-a978-2c1a7612a363 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:33.507766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d401fa2-c553-4f47-a978-2c1a7612a363 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:33.508476 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d401fa2-c553-4f47-a978-2c1a7612a363 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:33.508860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d401fa2-c553-4f47-a978-2c1a7612a363 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:33.522065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d401fa2-c553-4f47-a978-2c1a7612a363 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:33.524350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d401fa2-c553-4f47-a978-2c1a7612a363 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:33.541967 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1393/2781] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:12:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:12:33.633812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ff71c-85ad-4a9c-8291-c95011715502 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:33.641641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:33.642932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:33.643533 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:33.644122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] PATH_INFO: `/v3/projects/2a5cc1d2dee4412eb37b794156fdcb83` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:33.646083 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a5cc1d2dee4412eb37b794156fdcb83)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:33.650003 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:33.721104 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a5cc1d2dee4412eb37b794156fdcb83', 'request_id': 'req-80a56251-cf99-47c7-9738-8102467bc742'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:33.728449 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a5cc1d2dee4412eb37b794156fdcb83', 'request_id': 'req-80a56251-cf99-47c7-9738-8102467bc742'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:33.742081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a5cc1d2dee4412eb37b794156fdcb83', 'request_id': 'req-80a56251-cf99-47c7-9738-8102467bc742'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:33.764036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] The token cache is being invalidate because project 2a5cc1d2dee4412eb37b794156fdcb83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:12:33.764270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:33.765057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:33.767401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1389/2782] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:12:33 2022] DELETE /identity/v3/projects/2a5cc1d2dee4412eb37b794156fdcb83 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:12:33.771950 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d401fa2-c553-4f47-a978-2c1a7612a363 service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:33.779427 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:12:33.789243 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d401fa2-c553-4f47-a978-2c1a7612a363 service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:33.931831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:33.933992 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:33.935018 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:33.935422 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] PATH_INFO: `/v3/users/7b401a69e1bb4535af245ca77f59717f` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:33.937097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b401a69e1bb4535af245ca77f59717f)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:33.942200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:34.059412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b401a69e1bb4535af245ca77f59717f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:34.061861 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b401a69e1bb4535af245ca77f59717f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:34.084282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b401a69e1bb4535af245ca77f59717f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:34.115288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b401a69e1bb4535af245ca77f59717f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:34.138979 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-75163774-9ab8-4bb1-b8ad-8c5763d7bb19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b401a69e1bb4535af245ca77f59717f'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:34.164835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1394/2783] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:12:33 2022] DELETE /identity/v3/users/7b401a69e1bb4535af245ca77f59717f => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:12:34.635737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:34.638816 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:34.658059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a56251-cf99-47c7-9738-8102467bc742 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:34.756434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:34.757434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:34.757900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:34.758488 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] PATH_INFO: `/v3/projects/8b998a586ac34bff992e7d7f49d0bf18` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:34.760102 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b998a586ac34bff992e7d7f49d0bf18)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:34.762108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:34.765115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:12:34.863630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b998a586ac34bff992e7d7f49d0bf18', 'request_id': 'req-5848f96d-7849-49da-92cf-2039058ba26f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:34.871450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b998a586ac34bff992e7d7f49d0bf18', 'request_id': 'req-5848f96d-7849-49da-92cf-2039058ba26f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:34.882824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b998a586ac34bff992e7d7f49d0bf18', 'request_id': 'req-5848f96d-7849-49da-92cf-2039058ba26f'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:34.906424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] The token cache is being invalidate because project 8b998a586ac34bff992e7d7f49d0bf18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:12:34.907064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:34.907941 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:12:34.910394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1390/2784] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:12:34 2022] DELETE /identity/v3/projects/8b998a586ac34bff992e7d7f49d0bf18 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:12:35.506055 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80fc8e-c2c5-401c-b476-80464e52790b None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:35.506571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80fc8e-c2c5-401c-b476-80464e52790b None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:35.506669 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80fc8e-c2c5-401c-b476-80464e52790b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:35.535928 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-0d80fc8e-c2c5-401c-b476-80464e52790b None None] MFA Rules not processed for user `d19967d791b94e0eac324065e8e2f232`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:12:35.540325 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0d80fc8e-c2c5-401c-b476-80464e52790b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:35.606146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1395/2785] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:12:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:12:35.967062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:12:35.968999 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:36.039194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848f96d-7849-49da-92cf-2039058ba26f admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:12:36.043030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82979c10-3e89-4500-843f-ebb1e2a932b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:12:36.043683 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82979c10-3e89-4500-843f-ebb1e2a932b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:12:36.043951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82979c10-3e89-4500-843f-ebb1e2a932b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:12:36.044227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82979c10-3e89-4500-843f-ebb1e2a932b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:12:36.046179 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-82979c10-3e89-4500-843f-ebb1e2a932b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:12:36.057467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82979c10-3e89-4500-843f-ebb1e2a932b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:12:36.058935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82979c10-3e89-4500-843f-ebb1e2a932b0 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:12:36.068538 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1391/2786] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:12:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:12:45.599025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=76,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:12:46.064194 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:13:33.011570 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254933cb-d086-41f1-a18e-6628036a3f27 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:13:33.012665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254933cb-d086-41f1-a18e-6628036a3f27 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:13:33.012665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254933cb-d086-41f1-a18e-6628036a3f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:13:33.034982 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-254933cb-d086-41f1-a18e-6628036a3f27 None None] MFA Rules not processed for user `d19967d791b94e0eac324065e8e2f232`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:13:33.036273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-254933cb-d086-41f1-a18e-6628036a3f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:13:33.044897 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1396/2787] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:13:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:13:34.492573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82979c10-3e89-4500-843f-ebb1e2a932b0 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:13:34.494215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-82979c10-3e89-4500-843f-ebb1e2a932b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:13:34.593840 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82979c10-3e89-4500-843f-ebb1e2a932b0 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:13:34.599369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a830ca-0b0f-4ea2-aaa1-0202300bdb68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:13:34.600005 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a830ca-0b0f-4ea2-aaa1-0202300bdb68 service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:13:34.600311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a830ca-0b0f-4ea2-aaa1-0202300bdb68 service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:13:34.600590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a830ca-0b0f-4ea2-aaa1-0202300bdb68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:13:34.602478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a0a830ca-0b0f-4ea2-aaa1-0202300bdb68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:13:34.620634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a830ca-0b0f-4ea2-aaa1-0202300bdb68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:13:34.623172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a830ca-0b0f-4ea2-aaa1-0202300bdb68 service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:13:34.638301 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1392/2788] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:13:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:13:43.028012 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:13:44.631225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=41 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:14:27.690368 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b52f3b-4c5b-4f3b-84a7-a6627c980be2 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:14:27.690932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b52f3b-4c5b-4f3b-84a7-a6627c980be2 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:14:27.691113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b52f3b-4c5b-4f3b-84a7-a6627c980be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:14:27.712245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-07b52f3b-4c5b-4f3b-84a7-a6627c980be2 None None] MFA Rules not processed for user `11c07e105d1e420faa1b4b2be7dbe09c`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:14:27.713482 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-07b52f3b-4c5b-4f3b-84a7-a6627c980be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:14:27.776481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1397/2789] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:14:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:14:33.288728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb964-4abb-4785-8d2f-0887d88b3145 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:14:33.289304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb964-4abb-4785-8d2f-0887d88b3145 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:14:33.289304 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdeb964-4abb-4785-8d2f-0887d88b3145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:14:33.309581 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ebdeb964-4abb-4785-8d2f-0887d88b3145 None None] MFA Rules not processed for user `d19967d791b94e0eac324065e8e2f232`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:14:33.310831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ebdeb964-4abb-4785-8d2f-0887d88b3145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:14:33.318481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1393/2790] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:14:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:14:37.766170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=25 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:14:43.303763 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:15:25.216719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b52f3b-4c5b-4f3b-84a7-a6627c980be2 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:15:25.218398 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-07b52f3b-4c5b-4f3b-84a7-a6627c980be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:15:25.286544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b52f3b-4c5b-4f3b-84a7-a6627c980be2 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:15:25.290301 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def81804-66d0-4875-9621-c53e1921eab8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:15:25.290947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def81804-66d0-4875-9621-c53e1921eab8 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:15:25.291228 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def81804-66d0-4875-9621-c53e1921eab8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:15:25.291503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def81804-66d0-4875-9621-c53e1921eab8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:15:25.293488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-def81804-66d0-4875-9621-c53e1921eab8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:15:25.403795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def81804-66d0-4875-9621-c53e1921eab8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:15:25.406697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def81804-66d0-4875-9621-c53e1921eab8 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:15:25.438412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1398/2791] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:15:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:15:32.434917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeb964-4abb-4785-8d2f-0887d88b3145 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:15:32.450656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdeb964-4abb-4785-8d2f-0887d88b3145 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:15:32.456826 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdd064-69c4-4852-81cd-7007f4730a15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:15:32.457569 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdd064-69c4-4852-81cd-7007f4730a15 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:15:32.457780 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdd064-69c4-4852-81cd-7007f4730a15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:15:32.458117 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdd064-69c4-4852-81cd-7007f4730a15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:15:32.460802 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-27bdd064-69c4-4852-81cd-7007f4730a15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:15:32.476645 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bdd064-69c4-4852-81cd-7007f4730a15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:15:32.478139 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bdd064-69c4-4852-81cd-7007f4730a15 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:15:32.490253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1394/2792] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:15:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:15:35.434511 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=66 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:15:39.670209 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f6523-0a86-436a-93df-18b7eb9dc885 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:15:39.670957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f6523-0a86-436a-93df-18b7eb9dc885 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:15:39.670957 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f6523-0a86-436a-93df-18b7eb9dc885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:15:39.691383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-161f6523-0a86-436a-93df-18b7eb9dc885 None None] MFA Rules not processed for user `d19967d791b94e0eac324065e8e2f232`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:15:39.692625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-161f6523-0a86-436a-93df-18b7eb9dc885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:15:39.700407 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1399/2793] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:15:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:15:40.032026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdd064-69c4-4852-81cd-7007f4730a15 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:15:40.041582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdd064-69c4-4852-81cd-7007f4730a15 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:15:40.045403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917023f-3c41-42bb-abe8-359e8156df90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:15:40.046008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917023f-3c41-42bb-abe8-359e8156df90 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:15:40.046285 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917023f-3c41-42bb-abe8-359e8156df90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:15:40.046552 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917023f-3c41-42bb-abe8-359e8156df90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:15:40.048553 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e917023f-3c41-42bb-abe8-359e8156df90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:15:40.058519 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917023f-3c41-42bb-abe8-359e8156df90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:15:40.059937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917023f-3c41-42bb-abe8-359e8156df90 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:15:40.068613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1395/2794] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:15:40.432778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f6523-0a86-436a-93df-18b7eb9dc885 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:15:40.434548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-161f6523-0a86-436a-93df-18b7eb9dc885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:15:40.451688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f6523-0a86-436a-93df-18b7eb9dc885 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:15:40.457547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d2346-3258-4248-ae97-471eea33d261 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:15:40.457881 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d2346-3258-4248-ae97-471eea33d261 service placement] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:15:40.458587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d2346-3258-4248-ae97-471eea33d261 service placement] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:15:40.458900 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d2346-3258-4248-ae97-471eea33d261 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:15:40.461497 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a17d2346-3258-4248-ae97-471eea33d261 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:15:40.474287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d2346-3258-4248-ae97-471eea33d261 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:15:40.475879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d2346-3258-4248-ae97-471eea33d261 service placement] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:15:40.487731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1400/2795] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:15:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:15:42.227789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917023f-3c41-42bb-abe8-359e8156df90 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:15:42.238050 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917023f-3c41-42bb-abe8-359e8156df90 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:15:42.242363 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f46a1-a735-4c8f-b685-155e11b49039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:15:42.242996 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f46a1-a735-4c8f-b685-155e11b49039 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:15:42.243269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f46a1-a735-4c8f-b685-155e11b49039 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:15:42.243534 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f46a1-a735-4c8f-b685-155e11b49039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:15:42.245603 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-be0f46a1-a735-4c8f-b685-155e11b49039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:15:42.256220 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f46a1-a735-4c8f-b685-155e11b49039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:15:42.257752 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f46a1-a735-4c8f-b685-155e11b49039 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:15:42.266793 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1396/2796] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:15:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:15:50.482832 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=9 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:15:52.262864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=18 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:22.728776 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab5f0-271e-45a5-8f77-089e6537a801 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:22.729397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab5f0-271e-45a5-8f77-089e6537a801 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:22.729397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab5f0-271e-45a5-8f77-089e6537a801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:22.750803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c3cab5f0-271e-45a5-8f77-089e6537a801 None None] MFA Rules not processed for user `11c07e105d1e420faa1b4b2be7dbe09c`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:22.751942 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c3cab5f0-271e-45a5-8f77-089e6537a801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:22.760214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1401/2797] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:26.399547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f46a1-a735-4c8f-b685-155e11b49039 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:26.401296 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-be0f46a1-a735-4c8f-b685-155e11b49039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:26.425206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f46a1-a735-4c8f-b685-155e11b49039 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:26.433064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fd523-6ed8-4c14-9b5b-4394f1d3cda3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:26.433732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fd523-6ed8-4c14-9b5b-4394f1d3cda3 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:26.434017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fd523-6ed8-4c14-9b5b-4394f1d3cda3 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:26.434483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fd523-6ed8-4c14-9b5b-4394f1d3cda3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:26.436596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-333fd523-6ed8-4c14-9b5b-4394f1d3cda3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:26.452137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fd523-6ed8-4c14-9b5b-4394f1d3cda3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:26.454963 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fd523-6ed8-4c14-9b5b-4394f1d3cda3 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:26.471883 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1397/2798] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:16:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:16:32.744548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:36.464680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=6 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:38.115978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cab5f0-271e-45a5-8f77-089e6537a801 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:38.133698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cab5f0-271e-45a5-8f77-089e6537a801 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:38.140134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c780e9-0acd-46ea-9568-480c335177c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:38.141386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c780e9-0acd-46ea-9568-480c335177c9 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:38.141879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c780e9-0acd-46ea-9568-480c335177c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:38.142411 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c780e9-0acd-46ea-9568-480c335177c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:38.144583 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-43c780e9-0acd-46ea-9568-480c335177c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:38.159320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c780e9-0acd-46ea-9568-480c335177c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:38.161039 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c780e9-0acd-46ea-9568-480c335177c9 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:38.176379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1402/2799] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:16:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:16:46.979616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fd523-6ed8-4c14-9b5b-4394f1d3cda3 service nova] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:47.006480 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fd523-6ed8-4c14-9b5b-4394f1d3cda3 service nova] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:47.015338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a4181-bf30-4f0a-814d-598030e7dbd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:47.016459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a4181-bf30-4f0a-814d-598030e7dbd8 service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:47.016777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a4181-bf30-4f0a-814d-598030e7dbd8 service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:47.017054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a4181-bf30-4f0a-814d-598030e7dbd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:47.022546 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-e17a4181-bf30-4f0a-814d-598030e7dbd8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:47.036654 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a4181-bf30-4f0a-814d-598030e7dbd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:47.038438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a4181-bf30-4f0a-814d-598030e7dbd8 service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:47.054013 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1398/2800] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:16:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:16:47.069042 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c780e9-0acd-46ea-9568-480c335177c9 service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:47.072719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-43c780e9-0acd-46ea-9568-480c335177c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:47.161290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c780e9-0acd-46ea-9568-480c335177c9 service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:47.166304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:47.169306 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:47.169599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:47.172124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] PATH_INFO: `/v3/users/d19967d791b94e0eac324065e8e2f232` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:47.175649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d19967d791b94e0eac324065e8e2f232)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:47.177621 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:47.229978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd19967d791b94e0eac324065e8e2f232'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:47.230778 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd19967d791b94e0eac324065e8e2f232'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:47.240029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd19967d791b94e0eac324065e8e2f232'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:47.252809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd19967d791b94e0eac324065e8e2f232'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:47.259643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-5a807b08-fc23-4aca-afe6-5c4545300c1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd19967d791b94e0eac324065e8e2f232'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:47.265604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1403/2801] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:16:47 2022] DELETE /identity/v3/users/d19967d791b94e0eac324065e8e2f232 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:47.456476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a4181-bf30-4f0a-814d-598030e7dbd8 service placement] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:47.471168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a4181-bf30-4f0a-814d-598030e7dbd8 service placement] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:47.531438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:47.532007 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:47.532314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:47.532593 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] PATH_INFO: `/v3/projects/ac2e8512439547d9affb144ee0df1c39` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:47.533953 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac2e8512439547d9affb144ee0df1c39)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:47.535871 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:47.583335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac2e8512439547d9affb144ee0df1c39', 'request_id': 'req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:47.589783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac2e8512439547d9affb144ee0df1c39', 'request_id': 'req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:47.598769 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac2e8512439547d9affb144ee0df1c39', 'request_id': 'req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:47.610718 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] The token cache is being invalidate because project ac2e8512439547d9affb144ee0df1c39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:16:47.611006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:47.611700 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-a169fb55-1fc3-4f1a-a4f9-1fc68144d834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:47.613309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1399/2802] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:16:47 2022] DELETE /identity/v3/projects/ac2e8512439547d9affb144ee0df1c39 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:47.619584 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe4725-579a-40ac-a4bc-50bf3651765e None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:47.619894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe4725-579a-40ac-a4bc-50bf3651765e None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:47.620171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe4725-579a-40ac-a4bc-50bf3651765e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:47.641344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6cbe4725-579a-40ac-a4bc-50bf3651765e None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:47.642580 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6cbe4725-579a-40ac-a4bc-50bf3651765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:47.713312 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1404/2803] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:47.725678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae898e-697c-4e13-9bc6-e6ddd4723716 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:47.725917 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae898e-697c-4e13-9bc6-e6ddd4723716 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:47.726649 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae898e-697c-4e13-9bc6-e6ddd4723716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:47.754091 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-a7ae898e-697c-4e13-9bc6-e6ddd4723716 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:47.755329 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a7ae898e-697c-4e13-9bc6-e6ddd4723716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:47.766831 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1400/2804] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:47.773440 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe4725-579a-40ac-a4bc-50bf3651765e None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:47.785766 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe4725-579a-40ac-a4bc-50bf3651765e None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:47.791860 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db1b41-24d1-4408-b4fa-e376ca645f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:47.792550 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db1b41-24d1-4408-b4fa-e376ca645f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:47.792853 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db1b41-24d1-4408-b4fa-e376ca645f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:47.793109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db1b41-24d1-4408-b4fa-e376ca645f84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:47.793820 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db1b41-24d1-4408-b4fa-e376ca645f84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80164) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:16:47.794153 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db1b41-24d1-4408-b4fa-e376ca645f84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:47.795803 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db1b41-24d1-4408-b4fa-e376ca645f84 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:47.797552 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:47.809739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1405/2805] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:16:47.814287 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae898e-697c-4e13-9bc6-e6ddd4723716 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:47.824031 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae898e-697c-4e13-9bc6-e6ddd4723716 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:47.829401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e601c1-720b-4480-a82a-a44c0976da82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:47.830028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e601c1-720b-4480-a82a-a44c0976da82 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:47.830312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e601c1-720b-4480-a82a-a44c0976da82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:47.830597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e601c1-720b-4480-a82a-a44c0976da82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:47.831258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e601c1-720b-4480-a82a-a44c0976da82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:47.833268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e601c1-720b-4480-a82a-a44c0976da82 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:47.865849 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1401/2806] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:16:47 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:16:47.870402 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db1b41-24d1-4408-b4fa-e376ca645f84 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:47.881036 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db1b41-24d1-4408-b4fa-e376ca645f84 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:47.946639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df28c5-2c37-4cee-a572-a25508208487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:47.947276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df28c5-2c37-4cee-a572-a25508208487 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:47.947548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df28c5-2c37-4cee-a572-a25508208487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:47.947822 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df28c5-2c37-4cee-a572-a25508208487 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:47.948536 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df28c5-2c37-4cee-a572-a25508208487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:47.950481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df28c5-2c37-4cee-a572-a25508208487 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:47.973121 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1406/2807] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:16:47 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:16:47.977711 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e601c1-720b-4480-a82a-a44c0976da82 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:47.989157 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e601c1-720b-4480-a82a-a44c0976da82 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:47.994652 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549557e-2000-494a-b086-d9ac69c70621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:47.995272 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549557e-2000-494a-b086-d9ac69c70621 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:47.995588 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549557e-2000-494a-b086-d9ac69c70621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:47.995863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549557e-2000-494a-b086-d9ac69c70621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:47.996611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549557e-2000-494a-b086-d9ac69c70621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:47.998350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549557e-2000-494a-b086-d9ac69c70621 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:48.009105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1402/2808] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:16:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:16:48.013595 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df28c5-2c37-4cee-a572-a25508208487 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:48.024179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df28c5-2c37-4cee-a572-a25508208487 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:48.029348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76de46-78b7-4982-8885-b958289b15c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:48.029954 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76de46-78b7-4982-8885-b958289b15c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.030254 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76de46-78b7-4982-8885-b958289b15c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.030518 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76de46-78b7-4982-8885-b958289b15c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.031232 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76de46-78b7-4982-8885-b958289b15c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:48.032844 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76de46-78b7-4982-8885-b958289b15c9 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:48.043746 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1407/2809] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:16:48.048185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549557e-2000-494a-b086-d9ac69c70621 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:48.058590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549557e-2000-494a-b086-d9ac69c70621 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:48.063587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c02301-337d-49ea-ab7a-097a1546e2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:48.064256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c02301-337d-49ea-ab7a-097a1546e2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.064599 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c02301-337d-49ea-ab7a-097a1546e2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.064863 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c02301-337d-49ea-ab7a-097a1546e2f1 admin admin] PATH_INFO: `/v3/projects/73579419a77d4162a36980f275768897/users/2b7d9792bfcf46bebead6c20c76baac8/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.081137 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c02301-337d-49ea-ab7a-097a1546e2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73579419a77d4162a36980f275768897, user_id=2b7d9792bfcf46bebead6c20c76baac8, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:48.083821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c02301-337d-49ea-ab7a-097a1546e2f1 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:48.099531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1403/2810] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:16:48 2022] PUT /identity/v3/projects/73579419a77d4162a36980f275768897/users/2b7d9792bfcf46bebead6c20c76baac8/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:48.110530 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254360c-0bdf-4cb5-b701-a64dbd8c39fb None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.111041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254360c-0bdf-4cb5-b701-a64dbd8c39fb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.111159 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254360c-0bdf-4cb5-b701-a64dbd8c39fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.142888 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-9254360c-0bdf-4cb5-b701-a64dbd8c39fb None None] MFA Rules not processed for user `2b7d9792bfcf46bebead6c20c76baac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:48.144163 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9254360c-0bdf-4cb5-b701-a64dbd8c39fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:48.210129 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1408/2811] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:48.215008 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c02301-337d-49ea-ab7a-097a1546e2f1 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:48.231294 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:48.285565 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c02301-337d-49ea-ab7a-097a1546e2f1 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:48.290758 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25455dbf-b9c6-492e-b22d-054ed0fde999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:48.291376 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25455dbf-b9c6-492e-b22d-054ed0fde999 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.291650 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25455dbf-b9c6-492e-b22d-054ed0fde999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.291923 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25455dbf-b9c6-492e-b22d-054ed0fde999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.292615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25455dbf-b9c6-492e-b22d-054ed0fde999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:48.294609 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25455dbf-b9c6-492e-b22d-054ed0fde999 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:48.328129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1404/2812] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:16:48 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:16:48.332635 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254360c-0bdf-4cb5-b701-a64dbd8c39fb None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:48.343843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254360c-0bdf-4cb5-b701-a64dbd8c39fb None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:48.403651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922be19-5640-4498-94e0-7a10bc1492e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:48.404321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922be19-5640-4498-94e0-7a10bc1492e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.404604 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922be19-5640-4498-94e0-7a10bc1492e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.404880 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922be19-5640-4498-94e0-7a10bc1492e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.405553 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922be19-5640-4498-94e0-7a10bc1492e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:48.407526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922be19-5640-4498-94e0-7a10bc1492e3 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:48.417730 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:48.432984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1409/2813] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:16:48 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:16:48.437255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25455dbf-b9c6-492e-b22d-054ed0fde999 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:48.450530 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25455dbf-b9c6-492e-b22d-054ed0fde999 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:48.455482 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe1859-4b99-472e-987c-f7b8e1847782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:48.456357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe1859-4b99-472e-987c-f7b8e1847782 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.456529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe1859-4b99-472e-987c-f7b8e1847782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.456824 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe1859-4b99-472e-987c-f7b8e1847782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.457478 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe1859-4b99-472e-987c-f7b8e1847782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:48.459107 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe1859-4b99-472e-987c-f7b8e1847782 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:48.474175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1405/2814] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:16:48.479156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922be19-5640-4498-94e0-7a10bc1492e3 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:48.490670 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922be19-5640-4498-94e0-7a10bc1492e3 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:48.496415 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd5394-0468-49f1-8e69-acf15148e5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:48.497098 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5394-0468-49f1-8e69-acf15148e5a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.497379 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5394-0468-49f1-8e69-acf15148e5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.497652 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd5394-0468-49f1-8e69-acf15148e5a5 admin admin] PATH_INFO: `/v3/projects/d5077853343a4baca57840e77d12ecf1/users/695fa4f390ec49adbe17c0a24d76ebea/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.516025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd5394-0468-49f1-8e69-acf15148e5a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5077853343a4baca57840e77d12ecf1, user_id=695fa4f390ec49adbe17c0a24d76ebea, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:48.519549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd5394-0468-49f1-8e69-acf15148e5a5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:48.540652 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1410/2815] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:16:48 2022] PUT /identity/v3/projects/d5077853343a4baca57840e77d12ecf1/users/695fa4f390ec49adbe17c0a24d76ebea/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:48.541638 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe1859-4b99-472e-987c-f7b8e1847782 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:48.554234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe1859-4b99-472e-987c-f7b8e1847782 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:48.616614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968ac05-b06a-4332-bc0d-c590e4763b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:48.617256 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968ac05-b06a-4332-bc0d-c590e4763b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.617535 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968ac05-b06a-4332-bc0d-c590e4763b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.617812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968ac05-b06a-4332-bc0d-c590e4763b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.618742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968ac05-b06a-4332-bc0d-c590e4763b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:48.621573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968ac05-b06a-4332-bc0d-c590e4763b36 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:48.636944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1406/2816] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:16:48.638220 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd5394-0468-49f1-8e69-acf15148e5a5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:48.649452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd5394-0468-49f1-8e69-acf15148e5a5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:48.654639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146d66e-7300-4d79-b5da-b52319d27334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:48.655348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146d66e-7300-4d79-b5da-b52319d27334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.655624 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146d66e-7300-4d79-b5da-b52319d27334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.655894 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146d66e-7300-4d79-b5da-b52319d27334 admin admin] PATH_INFO: `/v3/projects/d5077853343a4baca57840e77d12ecf1/users/695fa4f390ec49adbe17c0a24d76ebea/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.658054 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146d66e-7300-4d79-b5da-b52319d27334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5077853343a4baca57840e77d12ecf1, user_id=695fa4f390ec49adbe17c0a24d76ebea, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:48.660739 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146d66e-7300-4d79-b5da-b52319d27334 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:48.672200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1411/2817] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:16:48 2022] PUT /identity/v3/projects/d5077853343a4baca57840e77d12ecf1/users/695fa4f390ec49adbe17c0a24d76ebea/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:48.684255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a861f30-d070-4c42-83aa-b34d9cfa610b None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.684812 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a861f30-d070-4c42-83aa-b34d9cfa610b None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.685177 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a861f30-d070-4c42-83aa-b34d9cfa610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.715255 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-0a861f30-d070-4c42-83aa-b34d9cfa610b None None] MFA Rules not processed for user `695fa4f390ec49adbe17c0a24d76ebea`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:48.716536 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0a861f30-d070-4c42-83aa-b34d9cfa610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:48.786038 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1407/2818] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 01 12:16:48.797949 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146d66e-7300-4d79-b5da-b52319d27334 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:48.799999 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b146d66e-7300-4d79-b5da-b52319d27334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:48.876136 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146d66e-7300-4d79-b5da-b52319d27334 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:48.879948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef411d-5971-496d-8f67-22216c01ebae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:48.880644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef411d-5971-496d-8f67-22216c01ebae service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:48.880902 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef411d-5971-496d-8f67-22216c01ebae service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:48.881200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef411d-5971-496d-8f67-22216c01ebae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:48.891883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef411d-5971-496d-8f67-22216c01ebae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:48.894005 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef411d-5971-496d-8f67-22216c01ebae service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:48.904620 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1412/2819] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:16:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:16:48.933560 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a861f30-d070-4c42-83aa-b34d9cfa610b None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:48.943710 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:49.000325 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a861f30-d070-4c42-83aa-b34d9cfa610b None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:49.004992 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:49.005644 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:49.005902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:49.006196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] PATH_INFO: `/v3/users/2b7d9792bfcf46bebead6c20c76baac8` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:49.007377 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b7d9792bfcf46bebead6c20c76baac8)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:49.009531 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:49.057664 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b7d9792bfcf46bebead6c20c76baac8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.070438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b7d9792bfcf46bebead6c20c76baac8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.077663 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b7d9792bfcf46bebead6c20c76baac8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.078346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b7d9792bfcf46bebead6c20c76baac8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.078942 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b7d9792bfcf46bebead6c20c76baac8'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.092748 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1408/2820] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:16:48 2022] DELETE /identity/v3/users/2b7d9792bfcf46bebead6c20c76baac8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:49.100043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73e469-d198-497e-adcc-f06d965e3fb9 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:49.100043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73e469-d198-497e-adcc-f06d965e3fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:49.100043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73e469-d198-497e-adcc-f06d965e3fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:49.118667 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-3e73e469-d198-497e-adcc-f06d965e3fb9 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:49.119873 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3e73e469-d198-497e-adcc-f06d965e3fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:49.184870 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1413/2821] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:49.195069 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:49.196589 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:49.256798 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b50010-a477-463f-b805-1d90539a2a4b admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:49.263597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343f0a0-5eb1-450e-bf46-55f2b9b5a3d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:49.264574 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343f0a0-5eb1-450e-bf46-55f2b9b5a3d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:49.264965 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343f0a0-5eb1-450e-bf46-55f2b9b5a3d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:49.265258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343f0a0-5eb1-450e-bf46-55f2b9b5a3d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:49.281728 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343f0a0-5eb1-450e-bf46-55f2b9b5a3d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:49.284054 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343f0a0-5eb1-450e-bf46-55f2b9b5a3d5 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:49.294218 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1409/2822] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:16:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:16:49.597021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73e469-d198-497e-adcc-f06d965e3fb9 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:49.598472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-3e73e469-d198-497e-adcc-f06d965e3fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:49.611595 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73e469-d198-497e-adcc-f06d965e3fb9 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:49.616904 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:49.617564 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:49.617842 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:49.618114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] PATH_INFO: `/v3/projects/73579419a77d4162a36980f275768897` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:49.619315 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73579419a77d4162a36980f275768897)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:49.621300 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:49.678077 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73579419a77d4162a36980f275768897', 'request_id': 'req-9da3d2ee-6b27-493c-8768-1974f2347e54'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.680074 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:49.684906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73579419a77d4162a36980f275768897', 'request_id': 'req-9da3d2ee-6b27-493c-8768-1974f2347e54'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.694017 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73579419a77d4162a36980f275768897', 'request_id': 'req-9da3d2ee-6b27-493c-8768-1974f2347e54'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.705200 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] The token cache is being invalidate because project 73579419a77d4162a36980f275768897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:16:49.705488 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.706282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.707824 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1414/2823] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:16:49 2022] DELETE /identity/v3/projects/73579419a77d4162a36980f275768897 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:49.712514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343f0a0-5eb1-450e-bf46-55f2b9b5a3d5 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:49.723314 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343f0a0-5eb1-450e-bf46-55f2b9b5a3d5 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:49.766190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:49.798307 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:49.798954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:49.799230 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:49.799499 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] PATH_INFO: `/v3/users/695fa4f390ec49adbe17c0a24d76ebea` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:49.800774 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] RBAC: Authorizing `identity:delete_user(user_id=695fa4f390ec49adbe17c0a24d76ebea)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:49.802742 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:49.856768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695fa4f390ec49adbe17c0a24d76ebea'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.870129 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695fa4f390ec49adbe17c0a24d76ebea'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.877242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695fa4f390ec49adbe17c0a24d76ebea'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.878020 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695fa4f390ec49adbe17c0a24d76ebea'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.878635 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-955becc4-cf0d-4207-8de3-ff7e9509485c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695fa4f390ec49adbe17c0a24d76ebea'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:49.892965 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1410/2824] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:16:49 2022] DELETE /identity/v3/users/695fa4f390ec49adbe17c0a24d76ebea => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:50.431065 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:50.432743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:50.448090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d2ee-6b27-493c-8768-1974f2347e54 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:50.509222 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:50.509901 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:50.510160 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:50.510432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] PATH_INFO: `/v3/projects/d5077853343a4baca57840e77d12ecf1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:50.511639 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5077853343a4baca57840e77d12ecf1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:50.513588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:50.563539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5077853343a4baca57840e77d12ecf1', 'request_id': 'req-4b76bc3c-531b-4874-bb30-f492bd2d7b50'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:50.569719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5077853343a4baca57840e77d12ecf1', 'request_id': 'req-4b76bc3c-531b-4874-bb30-f492bd2d7b50'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:50.579115 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5077853343a4baca57840e77d12ecf1', 'request_id': 'req-4b76bc3c-531b-4874-bb30-f492bd2d7b50'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:50.591373 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] The token cache is being invalidate because project d5077853343a4baca57840e77d12ecf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:16:50.591655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:50.592317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4b76bc3c-531b-4874-bb30-f492bd2d7b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:50.593875 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1415/2825] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:16:50 2022] DELETE /identity/v3/projects/d5077853343a4baca57840e77d12ecf1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:50.598975 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb779a9-2cfb-4589-a766-e51428ca9d5d None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:50.599497 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb779a9-2cfb-4589-a766-e51428ca9d5d None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:50.599616 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb779a9-2cfb-4589-a766-e51428ca9d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:50.623271 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-6eb779a9-2cfb-4589-a766-e51428ca9d5d None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:50.624545 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-6eb779a9-2cfb-4589-a766-e51428ca9d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:50.693094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1411/2826] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:50.701151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84691e6a-84e5-447a-b3fa-4ad16585a241 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:50.701436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84691e6a-84e5-447a-b3fa-4ad16585a241 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:50.701709 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84691e6a-84e5-447a-b3fa-4ad16585a241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:50.723282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-84691e6a-84e5-447a-b3fa-4ad16585a241 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:50.724522 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-84691e6a-84e5-447a-b3fa-4ad16585a241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:50.732920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1416/2827] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:50.737576 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb779a9-2cfb-4589-a766-e51428ca9d5d None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:50.748573 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb779a9-2cfb-4589-a766-e51428ca9d5d None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:50.753937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d7058-89f0-4af4-bcb9-acf9f115b3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:50.754674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d7058-89f0-4af4-bcb9-acf9f115b3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:50.754945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d7058-89f0-4af4-bcb9-acf9f115b3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:50.755210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d7058-89f0-4af4-bcb9-acf9f115b3c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:50.755909 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d7058-89f0-4af4-bcb9-acf9f115b3c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:16:50.756206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d7058-89f0-4af4-bcb9-acf9f115b3c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:50.757834 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d7058-89f0-4af4-bcb9-acf9f115b3c2 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:50.769876 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1412/2828] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 01 12:16:50.774906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84691e6a-84e5-447a-b3fa-4ad16585a241 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:50.788029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84691e6a-84e5-447a-b3fa-4ad16585a241 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:50.793495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee036fd-fa49-4129-95a4-597e34d7d7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:50.794116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee036fd-fa49-4129-95a4-597e34d7d7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:50.794386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee036fd-fa49-4129-95a4-597e34d7d7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:50.794658 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee036fd-fa49-4129-95a4-597e34d7d7e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:50.795328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee036fd-fa49-4129-95a4-597e34d7d7e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:50.797317 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee036fd-fa49-4129-95a4-597e34d7d7e4 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:50.830265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1417/2829] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:16:50 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:16:50.834567 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d7058-89f0-4af4-bcb9-acf9f115b3c2 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:50.845614 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d7058-89f0-4af4-bcb9-acf9f115b3c2 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:50.881873 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:50.907476 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baff2f9-0e27-47ef-8ad6-e85ea41d860d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:50.908099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baff2f9-0e27-47ef-8ad6-e85ea41d860d admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:50.908406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baff2f9-0e27-47ef-8ad6-e85ea41d860d admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:50.908684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baff2f9-0e27-47ef-8ad6-e85ea41d860d admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:50.909351 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baff2f9-0e27-47ef-8ad6-e85ea41d860d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:50.911279 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baff2f9-0e27-47ef-8ad6-e85ea41d860d admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:50.937138 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1413/2830] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:16:50 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:16:50.941525 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee036fd-fa49-4129-95a4-597e34d7d7e4 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:50.952253 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee036fd-fa49-4129-95a4-597e34d7d7e4 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:50.958177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551835d-b578-4eb6-846d-c29a455b4640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:50.958899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551835d-b578-4eb6-846d-c29a455b4640 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:50.959175 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551835d-b578-4eb6-846d-c29a455b4640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:50.959438 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551835d-b578-4eb6-846d-c29a455b4640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:50.960132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551835d-b578-4eb6-846d-c29a455b4640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:50.961779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551835d-b578-4eb6-846d-c29a455b4640 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:50.972381 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1418/2831] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:16:50.976628 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baff2f9-0e27-47ef-8ad6-e85ea41d860d admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:50.987500 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baff2f9-0e27-47ef-8ad6-e85ea41d860d admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:50.993770 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714c349-23ec-4362-b733-63e2077e6450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:50.994391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714c349-23ec-4362-b733-63e2077e6450 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:50.994667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714c349-23ec-4362-b733-63e2077e6450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:50.994940 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714c349-23ec-4362-b733-63e2077e6450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:50.995629 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714c349-23ec-4362-b733-63e2077e6450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:50.997278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714c349-23ec-4362-b733-63e2077e6450 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:51.008781 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1414/2832] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:16:51.013224 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551835d-b578-4eb6-846d-c29a455b4640 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:51.024214 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551835d-b578-4eb6-846d-c29a455b4640 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:51.030184 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328061-ef7b-4d72-88cd-9453da6b4e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:51.030878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328061-ef7b-4d72-88cd-9453da6b4e2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:51.031131 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328061-ef7b-4d72-88cd-9453da6b4e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:51.031403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db328061-ef7b-4d72-88cd-9453da6b4e2c admin admin] PATH_INFO: `/v3/projects/7a76dd008dd64f9e9eae26c6dbfe9ed3/users/1b643eb667764523b5adddeac2d63e56/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:51.046657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db328061-ef7b-4d72-88cd-9453da6b4e2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a76dd008dd64f9e9eae26c6dbfe9ed3, user_id=1b643eb667764523b5adddeac2d63e56, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:51.049382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db328061-ef7b-4d72-88cd-9453da6b4e2c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:51.066687 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1419/2833] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:16:51 2022] PUT /identity/v3/projects/7a76dd008dd64f9e9eae26c6dbfe9ed3/users/1b643eb667764523b5adddeac2d63e56/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:51.077586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e7aa0-9fa3-4d5a-b4c8-5c62efc6d458 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:51.077938 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e7aa0-9fa3-4d5a-b4c8-5c62efc6d458 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:51.078154 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e7aa0-9fa3-4d5a-b4c8-5c62efc6d458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:51.110745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-ee5e7aa0-9fa3-4d5a-b4c8-5c62efc6d458 None None] MFA Rules not processed for user `1b643eb667764523b5adddeac2d63e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:51.112270 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ee5e7aa0-9fa3-4d5a-b4c8-5c62efc6d458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:51.183878 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1415/2834] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:51 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:51.189019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328061-ef7b-4d72-88cd-9453da6b4e2c admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:51.201244 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:51.260195 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db328061-ef7b-4d72-88cd-9453da6b4e2c admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:51.265461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362f31a-013f-45ba-9951-c9faf5a888f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:51.266080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362f31a-013f-45ba-9951-c9faf5a888f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:51.266362 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362f31a-013f-45ba-9951-c9faf5a888f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:51.266640 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362f31a-013f-45ba-9951-c9faf5a888f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:51.267302 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362f31a-013f-45ba-9951-c9faf5a888f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:51.269262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362f31a-013f-45ba-9951-c9faf5a888f5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:51.297983 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1420/2835] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:16:51 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:16:51.302431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e7aa0-9fa3-4d5a-b4c8-5c62efc6d458 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:51.314984 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e7aa0-9fa3-4d5a-b4c8-5c62efc6d458 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:51.375821 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad016a3-c0fa-489f-9ba5-bb88da61085e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:51.376388 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad016a3-c0fa-489f-9ba5-bb88da61085e admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:51.376674 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad016a3-c0fa-489f-9ba5-bb88da61085e admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:51.376945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad016a3-c0fa-489f-9ba5-bb88da61085e admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:51.377622 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad016a3-c0fa-489f-9ba5-bb88da61085e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:51.379559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad016a3-c0fa-489f-9ba5-bb88da61085e admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:51.403422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1416/2836] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:16:51 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:16:51.408501 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362f31a-013f-45ba-9951-c9faf5a888f5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:51.419539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362f31a-013f-45ba-9951-c9faf5a888f5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:51.424378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39469022-c267-4830-8aec-ab6dfea295f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:51.424996 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39469022-c267-4830-8aec-ab6dfea295f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:51.425275 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39469022-c267-4830-8aec-ab6dfea295f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:51.425551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39469022-c267-4830-8aec-ab6dfea295f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:51.426283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39469022-c267-4830-8aec-ab6dfea295f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:51.427931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39469022-c267-4830-8aec-ab6dfea295f6 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:51.438609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1421/2837] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:16:51.443092 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad016a3-c0fa-489f-9ba5-bb88da61085e admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:51.453537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad016a3-c0fa-489f-9ba5-bb88da61085e admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:51.458882 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57a8a-5338-434d-8368-691ddb05d55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:51.459547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57a8a-5338-434d-8368-691ddb05d55f admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:51.459843 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57a8a-5338-434d-8368-691ddb05d55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:51.460106 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef57a8a-5338-434d-8368-691ddb05d55f admin admin] PATH_INFO: `/v3/projects/8cbf5ea52616407290143c0c09a5f7fe/users/f66e36335a7e4616a549ca72197a6e15/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:51.475935 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef57a8a-5338-434d-8368-691ddb05d55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbf5ea52616407290143c0c09a5f7fe, user_id=f66e36335a7e4616a549ca72197a6e15, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:51.478655 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef57a8a-5338-434d-8368-691ddb05d55f admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:51.494182 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1417/2838] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:16:51 2022] PUT /identity/v3/projects/8cbf5ea52616407290143c0c09a5f7fe/users/f66e36335a7e4616a549ca72197a6e15/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:51.498327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39469022-c267-4830-8aec-ab6dfea295f6 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:51.509472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39469022-c267-4830-8aec-ab6dfea295f6 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:51.570653 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63587b51-4de4-43bb-a48c-cfa5c771a86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:51.571282 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63587b51-4de4-43bb-a48c-cfa5c771a86c admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:51.571559 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63587b51-4de4-43bb-a48c-cfa5c771a86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:51.571868 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63587b51-4de4-43bb-a48c-cfa5c771a86c admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:51.572611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63587b51-4de4-43bb-a48c-cfa5c771a86c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:51.574226 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63587b51-4de4-43bb-a48c-cfa5c771a86c admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:51.586046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1422/2839] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 01 12:16:51.591494 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57a8a-5338-434d-8368-691ddb05d55f admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:51.605100 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef57a8a-5338-434d-8368-691ddb05d55f admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:51.614693 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8583b4-7ebf-430d-b638-1675bebaee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:51.615459 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8583b4-7ebf-430d-b638-1675bebaee0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:51.615733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8583b4-7ebf-430d-b638-1675bebaee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:51.616019 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8583b4-7ebf-430d-b638-1675bebaee0d admin admin] PATH_INFO: `/v3/projects/8cbf5ea52616407290143c0c09a5f7fe/users/f66e36335a7e4616a549ca72197a6e15/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:51.619394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8583b4-7ebf-430d-b638-1675bebaee0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbf5ea52616407290143c0c09a5f7fe, user_id=f66e36335a7e4616a549ca72197a6e15, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:51.622501 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8583b4-7ebf-430d-b638-1675bebaee0d admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:51.629057 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:51.635044 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1418/2840] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:16:51 2022] PUT /identity/v3/projects/8cbf5ea52616407290143c0c09a5f7fe/users/f66e36335a7e4616a549ca72197a6e15/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:51.706899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc8dd-6030-4304-82f0-5bd5ed52aff9 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:51.707169 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc8dd-6030-4304-82f0-5bd5ed52aff9 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:51.707424 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc8dd-6030-4304-82f0-5bd5ed52aff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:51.751326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-fc2cc8dd-6030-4304-82f0-5bd5ed52aff9 None None] MFA Rules not processed for user `f66e36335a7e4616a549ca72197a6e15`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:51.752587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-fc2cc8dd-6030-4304-82f0-5bd5ed52aff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:51.828097 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1423/2841] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:51 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:51.839828 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8583b4-7ebf-430d-b638-1675bebaee0d admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:51.841684 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0f8583b4-7ebf-430d-b638-1675bebaee0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:51.902166 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8583b4-7ebf-430d-b638-1675bebaee0d admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:51.906306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3ac45-34ce-457b-ac6c-6bd822980ad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:51.906932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3ac45-34ce-457b-ac6c-6bd822980ad6 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:51.907207 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3ac45-34ce-457b-ac6c-6bd822980ad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:51.907481 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3ac45-34ce-457b-ac6c-6bd822980ad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:51.918888 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3ac45-34ce-457b-ac6c-6bd822980ad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:51.920437 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b3ac45-34ce-457b-ac6c-6bd822980ad6 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:51.931216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1419/2842] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:16:51 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:16:52.028733 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc8dd-6030-4304-82f0-5bd5ed52aff9 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:52.036113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:52.045116 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc8dd-6030-4304-82f0-5bd5ed52aff9 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:52.051288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d47917-fde5-4eb0-9efd-45e56f34a56a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:52.052348 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d47917-fde5-4eb0-9efd-45e56f34a56a service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:52.052770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d47917-fde5-4eb0-9efd-45e56f34a56a service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:52.053113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d47917-fde5-4eb0-9efd-45e56f34a56a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:52.117828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d47917-fde5-4eb0-9efd-45e56f34a56a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:52.119280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d47917-fde5-4eb0-9efd-45e56f34a56a service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:52.128413 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1424/2843] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:16:52 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:16:57.044733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3ac45-34ce-457b-ac6c-6bd822980ad6 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:57.108746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3ac45-34ce-457b-ac6c-6bd822980ad6 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:57.113464 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:57.114820 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:57.114993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:57.115191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] PATH_INFO: `/v3/users/1b643eb667764523b5adddeac2d63e56` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:57.116560 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b643eb667764523b5adddeac2d63e56)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:57.118783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:57.169466 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b643eb667764523b5adddeac2d63e56'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.181954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b643eb667764523b5adddeac2d63e56'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.189368 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b643eb667764523b5adddeac2d63e56'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.190112 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b643eb667764523b5adddeac2d63e56'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.190775 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b643eb667764523b5adddeac2d63e56'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.205435 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1420/2844] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:16:57 2022] DELETE /identity/v3/users/1b643eb667764523b5adddeac2d63e56 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:57.210810 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b94fa1-cd4c-4a78-ac01-506c1f94a3eb None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:57.211334 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b94fa1-cd4c-4a78-ac01-506c1f94a3eb None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:57.211433 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b94fa1-cd4c-4a78-ac01-506c1f94a3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:57.236883 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-62b94fa1-cd4c-4a78-ac01-506c1f94a3eb None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:57.238155 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-62b94fa1-cd4c-4a78-ac01-506c1f94a3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:57.311715 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1425/2845] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:57.321733 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:57.323238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:57.346385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:57.387327 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cebe3-aa0b-4480-a771-e6dfd2b6edcc admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:57.391206 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd51e12-1eba-4465-b26d-246daefcf622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:57.391837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd51e12-1eba-4465-b26d-246daefcf622 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:57.392114 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd51e12-1eba-4465-b26d-246daefcf622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:57.392414 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd51e12-1eba-4465-b26d-246daefcf622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:57.401805 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd51e12-1eba-4465-b26d-246daefcf622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:57.403240 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd51e12-1eba-4465-b26d-246daefcf622 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:57.412936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1421/2846] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:16:57.699542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b94fa1-cd4c-4a78-ac01-506c1f94a3eb None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:57.701056 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-62b94fa1-cd4c-4a78-ac01-506c1f94a3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:57.713370 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b94fa1-cd4c-4a78-ac01-506c1f94a3eb None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:57.719046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:57.719693 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:57.720001 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:57.720285 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] PATH_INFO: `/v3/projects/7a76dd008dd64f9e9eae26c6dbfe9ed3` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:57.721521 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a76dd008dd64f9e9eae26c6dbfe9ed3)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:57.723473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:57.776978 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a76dd008dd64f9e9eae26c6dbfe9ed3', 'request_id': 'req-6227c8f5-57b8-46f6-a4ad-430103360dad'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.783602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a76dd008dd64f9e9eae26c6dbfe9ed3', 'request_id': 'req-6227c8f5-57b8-46f6-a4ad-430103360dad'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.791156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a76dd008dd64f9e9eae26c6dbfe9ed3', 'request_id': 'req-6227c8f5-57b8-46f6-a4ad-430103360dad'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.803572 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] The token cache is being invalidate because project 7a76dd008dd64f9e9eae26c6dbfe9ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:16:57.803902 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.804613 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.806212 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1426/2847] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:16:57 2022] DELETE /identity/v3/projects/7a76dd008dd64f9e9eae26c6dbfe9ed3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:57.810355 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd51e12-1eba-4465-b26d-246daefcf622 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:57.820269 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd51e12-1eba-4465-b26d-246daefcf622 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:57.877604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:57.878253 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:57.878528 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:57.878799 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] PATH_INFO: `/v3/users/f66e36335a7e4616a549ca72197a6e15` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:57.879971 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66e36335a7e4616a549ca72197a6e15)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:57.881943 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:57.934736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66e36335a7e4616a549ca72197a6e15'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.947211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66e36335a7e4616a549ca72197a6e15'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.955081 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66e36335a7e4616a549ca72197a6e15'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.956172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66e36335a7e4616a549ca72197a6e15'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.957213 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-958c7864-a280-4b6c-a180-a5297bae9d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66e36335a7e4616a549ca72197a6e15'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:57.971647 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1422/2848] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:16:57 2022] DELETE /identity/v3/users/f66e36335a7e4616a549ca72197a6e15 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:58.263416 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:58.264977 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:58.278651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227c8f5-57b8-46f6-a4ad-430103360dad admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:58.300641 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:58.343256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:58.343909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:58.344236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:58.344544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] PATH_INFO: `/v3/projects/8cbf5ea52616407290143c0c09a5f7fe` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:58.345743 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbf5ea52616407290143c0c09a5f7fe)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:58.347690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:58.397477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbf5ea52616407290143c0c09a5f7fe', 'request_id': 'req-db362baf-5e3b-42c1-a4ec-7922477914ba'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:58.403952 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbf5ea52616407290143c0c09a5f7fe', 'request_id': 'req-db362baf-5e3b-42c1-a4ec-7922477914ba'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:58.413814 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbf5ea52616407290143c0c09a5f7fe', 'request_id': 'req-db362baf-5e3b-42c1-a4ec-7922477914ba'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:58.425166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] The token cache is being invalidate because project 8cbf5ea52616407290143c0c09a5f7fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:16:58.425432 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:58.426080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-db362baf-5e3b-42c1-a4ec-7922477914ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:16:58.427629 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1427/2849] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:16:58 2022] DELETE /identity/v3/projects/8cbf5ea52616407290143c0c09a5f7fe => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:58.433730 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acb8c5-54b2-425b-8be6-9fc842d2f245 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:58.434291 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acb8c5-54b2-425b-8be6-9fc842d2f245 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:58.434291 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acb8c5-54b2-425b-8be6-9fc842d2f245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:58.454997 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-20acb8c5-54b2-425b-8be6-9fc842d2f245 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:58.456364 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-20acb8c5-54b2-425b-8be6-9fc842d2f245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:58.496549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:58.521084 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1423/2850] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:58.529400 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9b12f-4d34-47b8-afed-e4c7376dab22 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:58.529688 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9b12f-4d34-47b8-afed-e4c7376dab22 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:58.529963 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9b12f-4d34-47b8-afed-e4c7376dab22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:58.552098 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-c3c9b12f-4d34-47b8-afed-e4c7376dab22 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:58.553498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c3c9b12f-4d34-47b8-afed-e4c7376dab22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:58.561812 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1428/2851] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:58.566278 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acb8c5-54b2-425b-8be6-9fc842d2f245 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:58.576590 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acb8c5-54b2-425b-8be6-9fc842d2f245 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:58.581331 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20eee2e-af10-46d8-99ff-99c533c57f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:58.581959 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20eee2e-af10-46d8-99ff-99c533c57f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:58.582233 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20eee2e-af10-46d8-99ff-99c533c57f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:58.582479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20eee2e-af10-46d8-99ff-99c533c57f15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:58.583168 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20eee2e-af10-46d8-99ff-99c533c57f15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:16:58.583467 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20eee2e-af10-46d8-99ff-99c533c57f15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:58.585131 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20eee2e-af10-46d8-99ff-99c533c57f15 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:58.596797 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1424/2852] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:16:58.601478 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9b12f-4d34-47b8-afed-e4c7376dab22 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:58.610929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9b12f-4d34-47b8-afed-e4c7376dab22 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:58.616114 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85ae77-ac2a-428d-83f0-9949ed662054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:58.616769 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ae77-ac2a-428d-83f0-9949ed662054 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:58.617048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ae77-ac2a-428d-83f0-9949ed662054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:58.617343 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85ae77-ac2a-428d-83f0-9949ed662054 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:58.618048 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85ae77-ac2a-428d-83f0-9949ed662054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:58.619998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85ae77-ac2a-428d-83f0-9949ed662054 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:58.647801 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1429/2853] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:16:58 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:16:58.652311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20eee2e-af10-46d8-99ff-99c533c57f15 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:58.662373 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20eee2e-af10-46d8-99ff-99c533c57f15 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:58.728868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56d141-c823-44d2-9da5-b76558f84166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:58.729401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d141-c823-44d2-9da5-b76558f84166 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:58.729680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d141-c823-44d2-9da5-b76558f84166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:58.729951 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56d141-c823-44d2-9da5-b76558f84166 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:58.730633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56d141-c823-44d2-9da5-b76558f84166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:58.732656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56d141-c823-44d2-9da5-b76558f84166 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:58.756152 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1425/2854] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:16:58 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:16:58.760744 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85ae77-ac2a-428d-83f0-9949ed662054 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:58.771877 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85ae77-ac2a-428d-83f0-9949ed662054 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:58.777350 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61faf85-633a-462f-9a11-5f0896af2065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:58.778016 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61faf85-633a-462f-9a11-5f0896af2065 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:58.778290 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61faf85-633a-462f-9a11-5f0896af2065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:58.778563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61faf85-633a-462f-9a11-5f0896af2065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:58.779262 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61faf85-633a-462f-9a11-5f0896af2065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:58.780916 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61faf85-633a-462f-9a11-5f0896af2065 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:58.791166 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1430/2855] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:16:58.796747 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56d141-c823-44d2-9da5-b76558f84166 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:58.811283 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56d141-c823-44d2-9da5-b76558f84166 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:58.819787 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b366628-cc72-4204-a893-e413d5101a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:58.820434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b366628-cc72-4204-a893-e413d5101a8c admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:58.820716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b366628-cc72-4204-a893-e413d5101a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:58.820979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b366628-cc72-4204-a893-e413d5101a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:58.821677 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b366628-cc72-4204-a893-e413d5101a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:58.823281 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b366628-cc72-4204-a893-e413d5101a8c admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:58.835524 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1426/2856] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:16:58.840187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61faf85-633a-462f-9a11-5f0896af2065 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:58.853326 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61faf85-633a-462f-9a11-5f0896af2065 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:58.860113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58983b-d00a-4aaa-9327-9000a50298fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:58.860835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58983b-d00a-4aaa-9327-9000a50298fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:58.861118 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58983b-d00a-4aaa-9327-9000a50298fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:58.861394 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58983b-d00a-4aaa-9327-9000a50298fc admin admin] PATH_INFO: `/v3/projects/52c1d7ddf8bc4b99b12c101398e5b4a9/users/018aa00fb142476fa8c0642f494efb00/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:58.878410 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58983b-d00a-4aaa-9327-9000a50298fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c1d7ddf8bc4b99b12c101398e5b4a9, user_id=018aa00fb142476fa8c0642f494efb00, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:58.881126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58983b-d00a-4aaa-9327-9000a50298fc admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:58.896428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1431/2857] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:16:58 2022] PUT /identity/v3/projects/52c1d7ddf8bc4b99b12c101398e5b4a9/users/018aa00fb142476fa8c0642f494efb00/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:16:58.902431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a86767-2004-456f-a036-5390a914d458 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:58.902736 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a86767-2004-456f-a036-5390a914d458 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:58.903012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a86767-2004-456f-a036-5390a914d458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:58.922527 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-23a86767-2004-456f-a036-5390a914d458 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:58.923754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-23a86767-2004-456f-a036-5390a914d458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:58.985794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1427/2858] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:58.996178 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58983b-d00a-4aaa-9327-9000a50298fc admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:58.998229 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-af58983b-d00a-4aaa-9327-9000a50298fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:59.058386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58983b-d00a-4aaa-9327-9000a50298fc admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:59.062499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839f4c7-3ca0-4019-a3c9-5704204f2381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:59.063126 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839f4c7-3ca0-4019-a3c9-5704204f2381 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:59.063403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839f4c7-3ca0-4019-a3c9-5704204f2381 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:59.063675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839f4c7-3ca0-4019-a3c9-5704204f2381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:59.070452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:59.074557 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839f4c7-3ca0-4019-a3c9-5704204f2381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:59.076103 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839f4c7-3ca0-4019-a3c9-5704204f2381 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:59.090175 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1432/2859] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:16:59.770227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ee571-d6cf-464e-b1a7-1b8317422df5 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:59.770845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ee571-d6cf-464e-b1a7-1b8317422df5 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:59.770845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ee571-d6cf-464e-b1a7-1b8317422df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:59.802402 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-9d8ee571-d6cf-464e-b1a7-1b8317422df5 None None] MFA Rules not processed for user `018aa00fb142476fa8c0642f494efb00`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:16:59.803921 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-9d8ee571-d6cf-464e-b1a7-1b8317422df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:59.849678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:16:59.878844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1428/2860] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:16:59.890020 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839f4c7-3ca0-4019-a3c9-5704204f2381 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:16:59.892024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-8839f4c7-3ca0-4019-a3c9-5704204f2381 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:16:59.953659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839f4c7-3ca0-4019-a3c9-5704204f2381 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:16:59.958939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cc41a-c2cb-401e-b746-52004b6cbd77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:16:59.959906 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cc41a-c2cb-401e-b746-52004b6cbd77 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:16:59.960427 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cc41a-c2cb-401e-b746-52004b6cbd77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:16:59.960704 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cc41a-c2cb-401e-b746-52004b6cbd77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:16:59.970465 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cc41a-c2cb-401e-b746-52004b6cbd77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:16:59.971920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cc41a-c2cb-401e-b746-52004b6cbd77 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:16:59.981235 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1433/2861] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:16:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:17:01.369587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073eeeb-bbe6-4551-a3f0-e44b99813bfe None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:17:01.370408 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073eeeb-bbe6-4551-a3f0-e44b99813bfe None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:17:01.370408 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073eeeb-bbe6-4551-a3f0-e44b99813bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:17:01.400118 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-5073eeeb-bbe6-4551-a3f0-e44b99813bfe None None] MFA Rules not processed for user `018aa00fb142476fa8c0642f494efb00`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:17:01.401374 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5073eeeb-bbe6-4551-a3f0-e44b99813bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:17:01.408864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1429/2862] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:17:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:17:09.976849 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=29 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:17:11.392155 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=13 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:18:04.256025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83ccf7-5adf-4dbb-87b4-d9d020060706 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:18:04.256643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83ccf7-5adf-4dbb-87b4-d9d020060706 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:18:04.256643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83ccf7-5adf-4dbb-87b4-d9d020060706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:18:04.277049 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-6f83ccf7-5adf-4dbb-87b4-d9d020060706 None None] MFA Rules not processed for user `018aa00fb142476fa8c0642f494efb00`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:18:04.278259 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6f83ccf7-5adf-4dbb-87b4-d9d020060706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:18:04.288892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1434/2863] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:18:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:18:14.271218 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=3 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:18:22.692753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073eeeb-bbe6-4551-a3f0-e44b99813bfe None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:18:22.695089 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5073eeeb-bbe6-4551-a3f0-e44b99813bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:18:22.708328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073eeeb-bbe6-4551-a3f0-e44b99813bfe None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:18:22.713147 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55970842-888d-4d03-aabf-e7c4670e34ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:18:22.713819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55970842-888d-4d03-aabf-e7c4670e34ac service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:18:22.714099 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55970842-888d-4d03-aabf-e7c4670e34ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:18:22.714366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55970842-888d-4d03-aabf-e7c4670e34ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:18:22.716276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-55970842-888d-4d03-aabf-e7c4670e34ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:18:22.786094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55970842-888d-4d03-aabf-e7c4670e34ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:18:22.788200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55970842-888d-4d03-aabf-e7c4670e34ac service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:18:22.798993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1430/2864] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:18:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:18:23.910156 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83ccf7-5adf-4dbb-87b4-d9d020060706 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:18:23.911770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-6f83ccf7-5adf-4dbb-87b4-d9d020060706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:18:24.010276 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83ccf7-5adf-4dbb-87b4-d9d020060706 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:18:24.016403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0cd6a-c82c-4cf9-abf3-9b0352c67104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:18:24.017051 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0cd6a-c82c-4cf9-abf3-9b0352c67104 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:18:24.017328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0cd6a-c82c-4cf9-abf3-9b0352c67104 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:18:24.017598 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0cd6a-c82c-4cf9-abf3-9b0352c67104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:18:24.019445 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-58c0cd6a-c82c-4cf9-abf3-9b0352c67104 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:18:24.033141 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0cd6a-c82c-4cf9-abf3-9b0352c67104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:18:24.039192 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c0cd6a-c82c-4cf9-abf3-9b0352c67104 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:18:24.054221 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1435/2865] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:18:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:18:32.794562 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=28 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:18:34.048831 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=34 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:18:59.614237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55970842-888d-4d03-aabf-e7c4670e34ac service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:18:59.617095 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-55970842-888d-4d03-aabf-e7c4670e34ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:18:59.710732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55970842-888d-4d03-aabf-e7c4670e34ac service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:18:59.719328 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bce16-c481-4ef8-b7b1-7a0f87720f73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:18:59.720320 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bce16-c481-4ef8-b7b1-7a0f87720f73 service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:18:59.720804 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bce16-c481-4ef8-b7b1-7a0f87720f73 service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:18:59.721259 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693bce16-c481-4ef8-b7b1-7a0f87720f73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:18:59.724276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-693bce16-c481-4ef8-b7b1-7a0f87720f73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:18:59.746330 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bce16-c481-4ef8-b7b1-7a0f87720f73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:18:59.749344 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693bce16-c481-4ef8-b7b1-7a0f87720f73 service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:18:59.764522 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1431/2866] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:18:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:19:09.759306 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=32 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:20:24.700502 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0cd6a-c82c-4cf9-abf3-9b0352c67104 service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:20:24.711456 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0cd6a-c82c-4cf9-abf3-9b0352c67104 service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:20:24.715947 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285efce-42db-40f7-83b1-940c0dcbbb76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:20:24.716617 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285efce-42db-40f7-83b1-940c0dcbbb76 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:20:24.716899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285efce-42db-40f7-83b1-940c0dcbbb76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:20:24.717168 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285efce-42db-40f7-83b1-940c0dcbbb76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:20:24.719365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f285efce-42db-40f7-83b1-940c0dcbbb76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:20:24.735729 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285efce-42db-40f7-83b1-940c0dcbbb76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:20:24.737245 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f285efce-42db-40f7-83b1-940c0dcbbb76 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:20:24.749971 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1436/2867] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:20:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:20:34.731778 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bce16-c481-4ef8-b7b1-7a0f87720f73 service placement] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:20:34.744204 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=6 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:20:34.746210 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693bce16-c481-4ef8-b7b1-7a0f87720f73 service placement] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:20:34.752074 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214493a-e7ab-4108-8489-3d8f442babda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:20:34.752932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214493a-e7ab-4108-8489-3d8f442babda service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:20:34.753302 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214493a-e7ab-4108-8489-3d8f442babda service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:20:34.753765 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214493a-e7ab-4108-8489-3d8f442babda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:20:34.756680 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-4214493a-e7ab-4108-8489-3d8f442babda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:20:34.773211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214493a-e7ab-4108-8489-3d8f442babda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:20:34.774701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214493a-e7ab-4108-8489-3d8f442babda service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:20:34.786727 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1432/2868] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:20:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:20:44.781036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=6 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:20:55.373711 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285efce-42db-40f7-83b1-940c0dcbbb76 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:20:55.383464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f285efce-42db-40f7-83b1-940c0dcbbb76 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:20:55.387772 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058a21a-e232-44f8-bf94-47cd72e3ba4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:20:55.388448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058a21a-e232-44f8-bf94-47cd72e3ba4d service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:20:55.388723 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058a21a-e232-44f8-bf94-47cd72e3ba4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:20:55.389013 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058a21a-e232-44f8-bf94-47cd72e3ba4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:20:55.391145 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-0058a21a-e232-44f8-bf94-47cd72e3ba4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:20:55.402203 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058a21a-e232-44f8-bf94-47cd72e3ba4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:20:55.403651 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058a21a-e232-44f8-bf94-47cd72e3ba4d service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:20:55.412473 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1437/2869] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:20:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:21:05.408586 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=6 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:22:00.621503 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214493a-e7ab-4108-8489-3d8f442babda service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:22:00.623215 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-4214493a-e7ab-4108-8489-3d8f442babda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:22:00.698225 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214493a-e7ab-4108-8489-3d8f442babda service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:22:00.703521 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d5ddc-dc34-460c-9a2c-3f5842a797e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:22:00.704150 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d5ddc-dc34-460c-9a2c-3f5842a797e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:22:00.704474 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d5ddc-dc34-460c-9a2c-3f5842a797e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:22:00.704749 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d5ddc-dc34-460c-9a2c-3f5842a797e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:22:00.732796 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d5ddc-dc34-460c-9a2c-3f5842a797e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:22:00.734277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2d5ddc-dc34-460c-9a2c-3f5842a797e2 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:22:00.744964 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1433/2870] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:22:00 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 01 12:22:10.739979 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=34 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:23:22.418545 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058a21a-e232-44f8-bf94-47cd72e3ba4d service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:23:22.452471 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058a21a-e232-44f8-bf94-47cd72e3ba4d service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:23:22.458804 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25d1b7-3869-4c63-afe1-7a4d749198d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:23:22.459428 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25d1b7-3869-4c63-afe1-7a4d749198d8 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:23:22.459708 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25d1b7-3869-4c63-afe1-7a4d749198d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:23:22.459984 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25d1b7-3869-4c63-afe1-7a4d749198d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:23:22.469100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25d1b7-3869-4c63-afe1-7a4d749198d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:23:22.470713 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c25d1b7-3869-4c63-afe1-7a4d749198d8 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:23:22.501320 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1438/2871] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:23:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:23:32.498086 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=21 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:24:01.628900 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d5ddc-dc34-460c-9a2c-3f5842a797e2 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:24:01.630514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-cc2d5ddc-dc34-460c-9a2c-3f5842a797e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:24:01.671261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d5ddc-dc34-460c-9a2c-3f5842a797e2 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:24:01.679219 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a52fa7-bccc-40ee-b76c-41c27a809f6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:24:01.680897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a52fa7-bccc-40ee-b76c-41c27a809f6f service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:24:01.681555 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a52fa7-bccc-40ee-b76c-41c27a809f6f service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:24:01.682615 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a52fa7-bccc-40ee-b76c-41c27a809f6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:24:01.692985 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a52fa7-bccc-40ee-b76c-41c27a809f6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:24:01.694621 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a52fa7-bccc-40ee-b76c-41c27a809f6f service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:24:01.705762 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1434/2872] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:24:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:24:11.701026 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=15 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:26:00.406646 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25d1b7-3869-4c63-afe1-7a4d749198d8 service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:26:00.427062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25d1b7-3869-4c63-afe1-7a4d749198d8 service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:26:00.431356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9b47f-94b7-4735-be2b-88088b9b74dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:26:00.432004 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9b47f-94b7-4735-be2b-88088b9b74dc service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:26:00.432280 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9b47f-94b7-4735-be2b-88088b9b74dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:26:00.432587 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea9b47f-94b7-4735-be2b-88088b9b74dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:26:00.442448 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9b47f-94b7-4735-be2b-88088b9b74dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:26:00.444483 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea9b47f-94b7-4735-be2b-88088b9b74dc service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:26:00.456665 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1439/2873] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:26:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:26:10.451493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=10 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:27:23.529431 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a52fa7-bccc-40ee-b76c-41c27a809f6f service placement] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:27:23.593757 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a52fa7-bccc-40ee-b76c-41c27a809f6f service placement] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:27:23.597928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc25d68-9564-47e1-b3b9-a2930ba16472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:27:23.598557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25d68-9564-47e1-b3b9-a2930ba16472 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:27:23.598813 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25d68-9564-47e1-b3b9-a2930ba16472 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:27:23.599094 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25d68-9564-47e1-b3b9-a2930ba16472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:27:23.601079 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-4dc25d68-9564-47e1-b3b9-a2930ba16472 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:27:23.685246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc25d68-9564-47e1-b3b9-a2930ba16472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:27:23.686636 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc25d68-9564-47e1-b3b9-a2930ba16472 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:27:23.705369 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1435/2874] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:27:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:27:33.702908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=61 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:28:27.240313 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9b47f-94b7-4735-be2b-88088b9b74dc service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:28:27.242839 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-9ea9b47f-94b7-4735-be2b-88088b9b74dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:28:27.262347 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea9b47f-94b7-4735-be2b-88088b9b74dc service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:28:27.267134 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5e898-7e3d-4943-84f6-b42050c2115d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:28:27.267756 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5e898-7e3d-4943-84f6-b42050c2115d service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:28:27.268035 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5e898-7e3d-4943-84f6-b42050c2115d service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:28:27.268412 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5e898-7e3d-4943-84f6-b42050c2115d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:28:27.270544 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a6c5e898-7e3d-4943-84f6-b42050c2115d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:28:27.350346 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5e898-7e3d-4943-84f6-b42050c2115d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:28:27.351822 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5e898-7e3d-4943-84f6-b42050c2115d service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:28:27.362321 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1440/2875] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:28:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:28:37.356845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=33 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:29:06.626707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc25d68-9564-47e1-b3b9-a2930ba16472 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:29:06.629375 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-4dc25d68-9564-47e1-b3b9-a2930ba16472 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:29:06.711869 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc25d68-9564-47e1-b3b9-a2930ba16472 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:29:06.718332 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161afe0f-3250-45eb-a61f-5ade3dd6132e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:29:06.718966 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161afe0f-3250-45eb-a61f-5ade3dd6132e service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:29:06.719248 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161afe0f-3250-45eb-a61f-5ade3dd6132e service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:29:06.719520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161afe0f-3250-45eb-a61f-5ade3dd6132e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:29:06.721642 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-161afe0f-3250-45eb-a61f-5ade3dd6132e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:29:06.734483 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161afe0f-3250-45eb-a61f-5ade3dd6132e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:29:06.736110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161afe0f-3250-45eb-a61f-5ade3dd6132e service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:29:06.747700 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1436/2876] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:29:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:29:16.742640 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=32 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:30:26.997439 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5e898-7e3d-4943-84f6-b42050c2115d service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:30:26.999030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a6c5e898-7e3d-4943-84f6-b42050c2115d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:30:27.089419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5e898-7e3d-4943-84f6-b42050c2115d service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:30:27.093090 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed07da-8936-4047-a50f-339047d3fc9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:30:27.093720 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed07da-8936-4047-a50f-339047d3fc9c service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:30:27.093998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed07da-8936-4047-a50f-339047d3fc9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:30:27.094267 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed07da-8936-4047-a50f-339047d3fc9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:30:27.174418 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed07da-8936-4047-a50f-339047d3fc9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:30:27.175794 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed07da-8936-4047-a50f-339047d3fc9c service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:30:27.197377 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1441/2877] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:30:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:30:28.406234 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161afe0f-3250-45eb-a61f-5ade3dd6132e service placement] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:30:28.416422 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161afe0f-3250-45eb-a61f-5ade3dd6132e service placement] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:30:28.420445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96461c52-6b40-4e88-a09c-187d2a225237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:30:28.422496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96461c52-6b40-4e88-a09c-187d2a225237 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:30:28.422496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96461c52-6b40-4e88-a09c-187d2a225237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:30:28.422496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96461c52-6b40-4e88-a09c-187d2a225237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:30:28.424444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-96461c52-6b40-4e88-a09c-187d2a225237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:30:28.454696 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96461c52-6b40-4e88-a09c-187d2a225237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:30:28.456237 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96461c52-6b40-4e88-a09c-187d2a225237 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:30:28.471792 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1437/2878] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:30:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:30:37.194021 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=66 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:30:38.464595 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=11 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:31:06.341043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed07da-8936-4047-a50f-339047d3fc9c service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:31:06.352142 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed07da-8936-4047-a50f-339047d3fc9c service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:31:06.356955 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0229c58-23a3-4b86-a7c5-20d194069e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:31:06.357628 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0229c58-23a3-4b86-a7c5-20d194069e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:31:06.357905 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0229c58-23a3-4b86-a7c5-20d194069e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:31:06.358177 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0229c58-23a3-4b86-a7c5-20d194069e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:31:06.360599 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b0229c58-23a3-4b86-a7c5-20d194069e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:31:06.374846 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0229c58-23a3-4b86-a7c5-20d194069e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:31:06.376341 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0229c58-23a3-4b86-a7c5-20d194069e49 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:31:06.385435 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1442/2879] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:31:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:31:16.381442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=6 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:32:23.569387 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96461c52-6b40-4e88-a09c-187d2a225237 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:32:23.572438 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-96461c52-6b40-4e88-a09c-187d2a225237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:32:23.622585 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96461c52-6b40-4e88-a09c-187d2a225237 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:32:23.628893 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ba4ae-d0a1-4146-b1a4-ce24e4ff95e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:32:23.629515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ba4ae-d0a1-4146-b1a4-ce24e4ff95e4 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:32:23.629789 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ba4ae-d0a1-4146-b1a4-ce24e4ff95e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:32:23.630064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ba4ae-d0a1-4146-b1a4-ce24e4ff95e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:32:23.638156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ba4ae-d0a1-4146-b1a4-ce24e4ff95e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:32:23.639739 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ba4ae-d0a1-4146-b1a4-ce24e4ff95e4 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:32:23.648868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1438/2880] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:32:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:32:33.644777 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=20 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:33:26.249339 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0229c58-23a3-4b86-a7c5-20d194069e49 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:33:26.251060 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b0229c58-23a3-4b86-a7c5-20d194069e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:33:26.328256 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0229c58-23a3-4b86-a7c5-20d194069e49 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:33:26.332151 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664fbca-e820-475a-a849-513528d39ddf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:33:26.332823 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664fbca-e820-475a-a849-513528d39ddf service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:33:26.333102 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664fbca-e820-475a-a849-513528d39ddf service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:33:26.333386 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664fbca-e820-475a-a849-513528d39ddf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:33:26.344690 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664fbca-e820-475a-a849-513528d39ddf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:33:26.346917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664fbca-e820-475a-a849-513528d39ddf service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:33:26.379607 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1443/2881] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:33:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:33:36.376526 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=35 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:34:11.631667 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ba4ae-d0a1-4146-b1a4-ce24e4ff95e4 service nova] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:34:11.671382 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ba4ae-d0a1-4146-b1a4-ce24e4ff95e4 service nova] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:34:11.676993 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2493-7132-4998-9c62-fdf2f7109a1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:34:11.677604 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2493-7132-4998-9c62-fdf2f7109a1f service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:34:11.677884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2493-7132-4998-9c62-fdf2f7109a1f service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:34:11.678156 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2493-7132-4998-9c62-fdf2f7109a1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:34:11.687125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2493-7132-4998-9c62-fdf2f7109a1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:34:11.688768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2493-7132-4998-9c62-fdf2f7109a1f service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:34:11.699200 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1439/2882] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:34:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:34:21.694723 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=18 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:34:28.737531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664fbca-e820-475a-a849-513528d39ddf service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:34:28.739287 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-5664fbca-e820-475a-a849-513528d39ddf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:34:28.756606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664fbca-e820-475a-a849-513528d39ddf service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:34:28.762951 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c189677-c005-4826-a374-1ab619b411fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:34:28.763588 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c189677-c005-4826-a374-1ab619b411fc service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:34:28.763869 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c189677-c005-4826-a374-1ab619b411fc service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:34:28.764146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c189677-c005-4826-a374-1ab619b411fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:34:28.766119 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7c189677-c005-4826-a374-1ab619b411fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:34:28.779843 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c189677-c005-4826-a374-1ab619b411fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:34:28.781543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c189677-c005-4826-a374-1ab619b411fc service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:34:28.793924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1444/2883] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:34:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:34:38.788706 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=6 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:35:29.099857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2493-7132-4998-9c62-fdf2f7109a1f service placement] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:35:29.111493 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2493-7132-4998-9c62-fdf2f7109a1f service placement] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:35:29.117211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb911ec-ab23-4ca5-9a3a-cac2ee0a9985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:35:29.118105 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb911ec-ab23-4ca5-9a3a-cac2ee0a9985 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:35:29.118514 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb911ec-ab23-4ca5-9a3a-cac2ee0a9985 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:35:29.118819 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb911ec-ab23-4ca5-9a3a-cac2ee0a9985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:35:29.121021 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1eb911ec-ab23-4ca5-9a3a-cac2ee0a9985 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:35:29.136496 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb911ec-ab23-4ca5-9a3a-cac2ee0a9985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:35:29.137905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb911ec-ab23-4ca5-9a3a-cac2ee0a9985 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:35:29.150412 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1440/2884] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:35:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:35:30.422187 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c189677-c005-4826-a374-1ab619b411fc service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:35:30.438074 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c189677-c005-4826-a374-1ab619b411fc service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:35:30.443492 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd3bdd-4c4d-4866-9011-d8da7ebc7e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:35:30.444113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd3bdd-4c4d-4866-9011-d8da7ebc7e7b service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:35:30.444477 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd3bdd-4c4d-4866-9011-d8da7ebc7e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:35:30.444642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd3bdd-4c4d-4866-9011-d8da7ebc7e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:35:30.457179 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd3bdd-4c4d-4866-9011-d8da7ebc7e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:35:30.458627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd3bdd-4c4d-4866-9011-d8da7ebc7e7b service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:35:30.470594 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1445/2885] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:35:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:35:39.144316 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=6 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:35:40.465519 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=6 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:36:13.351264 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb911ec-ab23-4ca5-9a3a-cac2ee0a9985 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:36:13.378172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb911ec-ab23-4ca5-9a3a-cac2ee0a9985 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:36:13.382931 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48204600-9c4b-4e0a-9b6a-b05bd8eb427d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:36:13.383549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204600-9c4b-4e0a-9b6a-b05bd8eb427d service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:36:13.383822 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204600-9c4b-4e0a-9b6a-b05bd8eb427d service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:36:13.384090 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48204600-9c4b-4e0a-9b6a-b05bd8eb427d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:36:13.391844 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48204600-9c4b-4e0a-9b6a-b05bd8eb427d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:36:13.393350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48204600-9c4b-4e0a-9b6a-b05bd8eb427d service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:36:13.401424 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1441/2886] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:36:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:36:23.397887 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=10 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:36:26.903910 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd3bdd-4c4d-4866-9011-d8da7ebc7e7b service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:36:26.917847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd3bdd-4c4d-4866-9011-d8da7ebc7e7b service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:36:26.923899 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9e58c-3b32-4901-822f-86b947f4a805 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:36:26.924595 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9e58c-3b32-4901-822f-86b947f4a805 service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:36:26.924879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9e58c-3b32-4901-822f-86b947f4a805 service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:36:26.925180 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9e58c-3b32-4901-822f-86b947f4a805 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:36:26.927286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-85a9e58c-3b32-4901-822f-86b947f4a805 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:36:26.941043 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9e58c-3b32-4901-822f-86b947f4a805 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:36:26.942791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9e58c-3b32-4901-822f-86b947f4a805 service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:36:26.955908 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1446/2887] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:36:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:36:36.950191 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=6 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:38:26.534455 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48204600-9c4b-4e0a-9b6a-b05bd8eb427d service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:38:26.549401 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48204600-9c4b-4e0a-9b6a-b05bd8eb427d service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:38:26.556337 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47101a52-38e9-4bb3-879a-db33ddd59661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:38:26.556968 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47101a52-38e9-4bb3-879a-db33ddd59661 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:38:26.557241 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47101a52-38e9-4bb3-879a-db33ddd59661 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:38:26.557495 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47101a52-38e9-4bb3-879a-db33ddd59661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:38:26.566199 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47101a52-38e9-4bb3-879a-db33ddd59661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:38:26.567764 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47101a52-38e9-4bb3-879a-db33ddd59661 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:38:26.577450 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1442/2888] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:38:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:38:28.683596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9e58c-3b32-4901-822f-86b947f4a805 service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:38:28.740921 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9e58c-3b32-4901-822f-86b947f4a805 service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:38:28.744919 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fc919-c703-470c-b2ef-ec19d7af6427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:38:28.745548 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fc919-c703-470c-b2ef-ec19d7af6427 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:38:28.745845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fc919-c703-470c-b2ef-ec19d7af6427 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:38:28.746100 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664fc919-c703-470c-b2ef-ec19d7af6427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:38:28.748124 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-664fc919-c703-470c-b2ef-ec19d7af6427 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:38:28.836182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fc919-c703-470c-b2ef-ec19d7af6427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:38:28.837948 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664fc919-c703-470c-b2ef-ec19d7af6427 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:38:28.867172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47101a52-38e9-4bb3-879a-db33ddd59661 service nova] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:38:28.868167 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1447/2889] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:38:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:38:28.868807 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-47101a52-38e9-4bb3-879a-db33ddd59661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:38:28.881336 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47101a52-38e9-4bb3-879a-db33ddd59661 service nova] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:38:28.885323 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97984328-a615-4377-b013-9fba72b701a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:38:28.885950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97984328-a615-4377-b013-9fba72b701a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:38:28.886217 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97984328-a615-4377-b013-9fba72b701a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:38:28.886479 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97984328-a615-4377-b013-9fba72b701a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:38:28.896390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97984328-a615-4377-b013-9fba72b701a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:38:28.897902 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97984328-a615-4377-b013-9fba72b701a7 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:38:28.907652 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1443/2890] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:38:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:38:38.865931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=61 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:38:38.903211 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=12 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:39:18.579365 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fc919-c703-470c-b2ef-ec19d7af6427 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:39:18.581915 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-664fc919-c703-470c-b2ef-ec19d7af6427 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:39:18.676758 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664fc919-c703-470c-b2ef-ec19d7af6427 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:39:18.682337 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dca6b5-cc54-4127-bf5d-a185afa72733 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:39:18.682974 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dca6b5-cc54-4127-bf5d-a185afa72733 service placement] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:39:18.683273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dca6b5-cc54-4127-bf5d-a185afa72733 service placement] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:39:18.683549 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dca6b5-cc54-4127-bf5d-a185afa72733 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:39:18.685495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-a9dca6b5-cc54-4127-bf5d-a185afa72733 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:39:18.697734 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dca6b5-cc54-4127-bf5d-a185afa72733 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:39:18.699357 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dca6b5-cc54-4127-bf5d-a185afa72733 service placement] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:39:18.710213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1448/2891] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:39:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:39:28.705929 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=37 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:40:29.987353 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97984328-a615-4377-b013-9fba72b701a7 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:40:29.989916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-97984328-a615-4377-b013-9fba72b701a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:40:30.070190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97984328-a615-4377-b013-9fba72b701a7 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:40:30.074223 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126fb21-9377-4630-84b1-3013a53a353f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:40:30.074845 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126fb21-9377-4630-84b1-3013a53a353f service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:40:30.075122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126fb21-9377-4630-84b1-3013a53a353f service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:40:30.075394 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7126fb21-9377-4630-84b1-3013a53a353f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:40:30.173597 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126fb21-9377-4630-84b1-3013a53a353f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:40:30.175447 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7126fb21-9377-4630-84b1-3013a53a353f service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:40:30.204701 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1444/2892] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:40:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:40:31.313175 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dca6b5-cc54-4127-bf5d-a185afa72733 service placement] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:40:31.328186 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dca6b5-cc54-4127-bf5d-a185afa72733 service placement] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:40:31.335217 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78907891-0a83-4eb1-b488-594cd7bfeab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:40:31.336014 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907891-0a83-4eb1-b488-594cd7bfeab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:40:31.336414 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907891-0a83-4eb1-b488-594cd7bfeab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:40:31.336781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78907891-0a83-4eb1-b488-594cd7bfeab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:40:31.340157 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-78907891-0a83-4eb1-b488-594cd7bfeab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:40:31.370896 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78907891-0a83-4eb1-b488-594cd7bfeab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:40:31.372340 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78907891-0a83-4eb1-b488-594cd7bfeab8 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:40:31.384150 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1449/2893] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:40:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:40:40.201937 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=66 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:40:41.378960 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=11 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:44:23.557687 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126fb21-9377-4630-84b1-3013a53a353f service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:44:23.559389 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-7126fb21-9377-4630-84b1-3013a53a353f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:44:23.639305 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7126fb21-9377-4630-84b1-3013a53a353f service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:44:23.645202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc476c-a7d0-4b28-9416-cf350d17b6c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:44:23.645833 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc476c-a7d0-4b28-9416-cf350d17b6c3 service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:44:23.646110 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc476c-a7d0-4b28-9416-cf350d17b6c3 service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:44:23.646384 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc476c-a7d0-4b28-9416-cf350d17b6c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:44:23.657441 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc476c-a7d0-4b28-9416-cf350d17b6c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:44:23.659630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc476c-a7d0-4b28-9416-cf350d17b6c3 service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:44:23.675315 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1445/2894] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:44:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:44:29.134079 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78907891-0a83-4eb1-b488-594cd7bfeab8 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:44:29.148557 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc476c-a7d0-4b28-9416-cf350d17b6c3 service placement] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:44:29.156338 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-4efc476c-a7d0-4b28-9416-cf350d17b6c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:44:29.184389 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78907891-0a83-4eb1-b488-594cd7bfeab8 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:44:29.190211 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95017a-16b2-4156-bfec-0147d80dc493 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:44:29.191011 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95017a-16b2-4156-bfec-0147d80dc493 service cinder] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:44:29.191163 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc476c-a7d0-4b28-9416-cf350d17b6c3 service placement] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:44:29.191328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95017a-16b2-4156-bfec-0147d80dc493 service cinder] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:44:29.191609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95017a-16b2-4156-bfec-0147d80dc493 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:44:29.200707 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662203d-9553-4590-8780-835ef80a8904 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dc0102a256934641a999dd091e6ba0b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:44:29.201399 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662203d-9553-4590-8780-835ef80a8904 service cinder] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:44:29.201668 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662203d-9553-4590-8780-835ef80a8904 service cinder] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:44:29.201952 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662203d-9553-4590-8780-835ef80a8904 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:44:29.205954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a662203d-9553-4590-8780-835ef80a8904 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:44:29.210370 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95017a-16b2-4156-bfec-0147d80dc493 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:44:29.212008 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95017a-16b2-4156-bfec-0147d80dc493 service cinder] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:44:29.222606 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1450/2895] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:44:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:44:29.313175 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662203d-9553-4590-8780-835ef80a8904 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:44:29.317142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662203d-9553-4590-8780-835ef80a8904 service cinder] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:44:29.340634 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1446/2896] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:44:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:44:39.217920 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=18 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:44:39.337859 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=82 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:48:29.773303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95017a-16b2-4156-bfec-0147d80dc493 service cinder] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:48:29.795208 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95017a-16b2-4156-bfec-0147d80dc493 service cinder] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:48:29.799236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7803f92-bdfb-4d68-8be7-bc6101e8dc52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:48:29.799863 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7803f92-bdfb-4d68-8be7-bc6101e8dc52 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:48:29.800158 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7803f92-bdfb-4d68-8be7-bc6101e8dc52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:48:29.800479 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7803f92-bdfb-4d68-8be7-bc6101e8dc52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:48:29.802937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d7803f92-bdfb-4d68-8be7-bc6101e8dc52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:48:29.905127 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7803f92-bdfb-4d68-8be7-bc6101e8dc52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:48:29.906571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7803f92-bdfb-4d68-8be7-bc6101e8dc52 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:48:29.926019 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1451/2897] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:48:33.112853 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662203d-9553-4590-8780-835ef80a8904 service cinder] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:48:33.125734 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662203d-9553-4590-8780-835ef80a8904 service cinder] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:48:33.131550 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232e732-7b56-4e30-9c76-b38effef41fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:48:33.132492 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232e732-7b56-4e30-9c76-b38effef41fe service placement] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:48:33.132857 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232e732-7b56-4e30-9c76-b38effef41fe service placement] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:48:33.133212 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232e732-7b56-4e30-9c76-b38effef41fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:48:33.135190 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-5232e732-7b56-4e30-9c76-b38effef41fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:48:33.152468 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232e732-7b56-4e30-9c76-b38effef41fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:48:33.154317 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232e732-7b56-4e30-9c76-b38effef41fe service placement] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:48:33.171231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1447/2898] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:48:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:48:36.399534 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7803f92-bdfb-4d68-8be7-bc6101e8dc52 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:48:36.401037 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d7803f92-bdfb-4d68-8be7-bc6101e8dc52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:48:36.415697 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7803f92-bdfb-4d68-8be7-bc6101e8dc52 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:48:36.422770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:48:36.423332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:48:36.423602 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:48:36.424080 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] PATH_INFO: `/v3/users/11c07e105d1e420faa1b4b2be7dbe09c` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:48:36.425335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c07e105d1e420faa1b4b2be7dbe09c)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:48:36.427272 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:48:36.478763 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c07e105d1e420faa1b4b2be7dbe09c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:36.479459 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c07e105d1e420faa1b4b2be7dbe09c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:36.488695 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c07e105d1e420faa1b4b2be7dbe09c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:36.504678 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c07e105d1e420faa1b4b2be7dbe09c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:36.512041 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c07e105d1e420faa1b4b2be7dbe09c'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:36.518068 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1452/2899] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:48:36 2022] DELETE /identity/v3/users/11c07e105d1e420faa1b4b2be7dbe09c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:48:36.714967 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232e732-7b56-4e30-9c76-b38effef41fe service placement] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:48:36.726783 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232e732-7b56-4e30-9c76-b38effef41fe service placement] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:48:36.794515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:48:36.795417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:48:36.795753 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:48:36.796111 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] PATH_INFO: `/v3/projects/d21aa81cdb8f40f5931615c0d85895a2` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:48:36.797950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d21aa81cdb8f40f5931615c0d85895a2)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:48:36.800971 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:48:36.864851 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd21aa81cdb8f40f5931615c0d85895a2', 'request_id': 'req-89e33148-b277-4a54-94a8-8c0bcd76decc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:36.871648 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd21aa81cdb8f40f5931615c0d85895a2', 'request_id': 'req-89e33148-b277-4a54-94a8-8c0bcd76decc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:36.881411 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd21aa81cdb8f40f5931615c0d85895a2', 'request_id': 'req-89e33148-b277-4a54-94a8-8c0bcd76decc'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:36.897068 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] The token cache is being invalidate because project d21aa81cdb8f40f5931615c0d85895a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:48:36.897361 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:36.898093 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:36.899626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1448/2900] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:48:36 2022] DELETE /identity/v3/projects/d21aa81cdb8f40f5931615c0d85895a2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:48:36.903850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:48:36.905328 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:48:36.918964 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf473-6ccf-4802-8d48-415a4848b431 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:48:36.944896 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:48:36.981958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:48:36.982611 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:48:36.982885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:48:36.983154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] PATH_INFO: `/v3/users/51474b8d57594305ac4ead352a29e7e5` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:48:36.997892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51474b8d57594305ac4ead352a29e7e5)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:48:36.999851 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:48:37.053498 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51474b8d57594305ac4ead352a29e7e5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:37.054154 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51474b8d57594305ac4ead352a29e7e5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:37.062717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51474b8d57594305ac4ead352a29e7e5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:37.074882 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51474b8d57594305ac4ead352a29e7e5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:37.082779 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51474b8d57594305ac4ead352a29e7e5'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:37.089436 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1453/2901] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:48:36 2022] DELETE /identity/v3/users/51474b8d57594305ac4ead352a29e7e5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:48:37.290297 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:48:37.291772 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:48:37.306657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e33148-b277-4a54-94a8-8c0bcd76decc admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:48:37.373986 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:48:37.374639 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:48:37.374916 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:48:37.375191 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] PATH_INFO: `/v3/projects/dac130aac76044aba7fbfec95a9e9d9f` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:48:37.387366 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dac130aac76044aba7fbfec95a9e9d9f)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:48:37.390428 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:48:37.405258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:48:37.445245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dac130aac76044aba7fbfec95a9e9d9f', 'request_id': 'req-073929f6-3cb2-4b30-b41c-26186550bda6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:37.452367 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dac130aac76044aba7fbfec95a9e9d9f', 'request_id': 'req-073929f6-3cb2-4b30-b41c-26186550bda6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:37.461754 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dac130aac76044aba7fbfec95a9e9d9f', 'request_id': 'req-073929f6-3cb2-4b30-b41c-26186550bda6'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:37.472657 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] The token cache is being invalidate because project dac130aac76044aba7fbfec95a9e9d9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:48:37.472936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:37.473591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:48:37.475172 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1449/2902] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:48:37 2022] DELETE /identity/v3/projects/dac130aac76044aba7fbfec95a9e9d9f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:48:47.085627 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:48:47.469226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:49:26.552619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:26.554675 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:26.645563 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999f76f-97d6-4075-803c-408b05afdf58 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:26.717356 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2052a3-6261-4a99-9612-b789b2c9b3cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:26.718024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2052a3-6261-4a99-9612-b789b2c9b3cf service placement] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:26.718303 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2052a3-6261-4a99-9612-b789b2c9b3cf service placement] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:26.718574 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2052a3-6261-4a99-9612-b789b2c9b3cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:26.720614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2f2052a3-6261-4a99-9612-b789b2c9b3cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:26.733626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2052a3-6261-4a99-9612-b789b2c9b3cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:26.735240 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2052a3-6261-4a99-9612-b789b2c9b3cf service placement] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:26.746286 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1454/2903] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:49:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:49:26.817418 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:26.819145 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:26.884429 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073929f6-3cb2-4b30-b41c-26186550bda6 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:26.888263 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef338f-df15-4566-bc2b-1ddb3881e0f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:26.888908 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef338f-df15-4566-bc2b-1ddb3881e0f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:26.889185 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef338f-df15-4566-bc2b-1ddb3881e0f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:26.889452 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef338f-df15-4566-bc2b-1ddb3881e0f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:26.891549 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ecef338f-df15-4566-bc2b-1ddb3881e0f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:26.954761 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef338f-df15-4566-bc2b-1ddb3881e0f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:26.956529 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef338f-df15-4566-bc2b-1ddb3881e0f1 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:26.966582 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1450/2904] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:49:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:49:27.425547 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2052a3-6261-4a99-9612-b789b2c9b3cf service placement] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:27.427182 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2f2052a3-6261-4a99-9612-b789b2c9b3cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:27.440891 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2052a3-6261-4a99-9612-b789b2c9b3cf service placement] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:27.446749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eface777-d747-440a-9f1e-7b9104e4b387 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:27.447378 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eface777-d747-440a-9f1e-7b9104e4b387 service placement] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:27.447654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eface777-d747-440a-9f1e-7b9104e4b387 service placement] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:27.447917 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eface777-d747-440a-9f1e-7b9104e4b387 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:27.450024 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-eface777-d747-440a-9f1e-7b9104e4b387 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:27.462987 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eface777-d747-440a-9f1e-7b9104e4b387 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:27.464623 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eface777-d747-440a-9f1e-7b9104e4b387 service placement] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:27.478216 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1455/2905] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:49:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:49:27.495540 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef338f-df15-4566-bc2b-1ddb3881e0f1 service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:27.497242 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-ecef338f-df15-4566-bc2b-1ddb3881e0f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:27.510732 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef338f-df15-4566-bc2b-1ddb3881e0f1 service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:27.516319 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:27.517000 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:27.517341 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:27.517671 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] PATH_INFO: `/v3/users/018aa00fb142476fa8c0642f494efb00` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:27.533444 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=018aa00fb142476fa8c0642f494efb00)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:27.535395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:27.580312 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018aa00fb142476fa8c0642f494efb00'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:49:27.593195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018aa00fb142476fa8c0642f494efb00'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:49:27.601359 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018aa00fb142476fa8c0642f494efb00'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:49:27.602059 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018aa00fb142476fa8c0642f494efb00'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:49:27.602708 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-9e57ed0f-a7ab-4595-aeda-9120cfc4cf14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018aa00fb142476fa8c0642f494efb00'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:49:27.621947 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1451/2906] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:49:27 2022] DELETE /identity/v3/users/018aa00fb142476fa8c0642f494efb00 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:49:27.800609 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eface777-d747-440a-9f1e-7b9104e4b387 service placement] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:27.816830 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eface777-d747-440a-9f1e-7b9104e4b387 service placement] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:27.883397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:27.884131 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:27.884442 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:27.884717 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] PATH_INFO: `/v3/projects/52c1d7ddf8bc4b99b12c101398e5b4a9` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:27.895791 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c1d7ddf8bc4b99b12c101398e5b4a9)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:27.897770 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:27.899288 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:49:27.948619 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c1d7ddf8bc4b99b12c101398e5b4a9', 'request_id': 'req-4543454d-eec0-47de-bb52-f6c56fe5de74'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:49:27.955719 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c1d7ddf8bc4b99b12c101398e5b4a9', 'request_id': 'req-4543454d-eec0-47de-bb52-f6c56fe5de74'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:49:27.964539 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c1d7ddf8bc4b99b12c101398e5b4a9', 'request_id': 'req-4543454d-eec0-47de-bb52-f6c56fe5de74'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:49:27.976986 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] The token cache is being invalidate because project 52c1d7ddf8bc4b99b12c101398e5b4a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80164) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:49:27.977270 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:49:27.977945 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-4543454d-eec0-47de-bb52-f6c56fe5de74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:49:27.979485 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1456/2907] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:49:27 2022] DELETE /identity/v3/projects/52c1d7ddf8bc4b99b12c101398e5b4a9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:49:27.985415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7b879-0896-4ad3-8a4d-a28eb0456e44 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:27.985956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7b879-0896-4ad3-8a4d-a28eb0456e44 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:27.985956 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7b879-0896-4ad3-8a4d-a28eb0456e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:28.003860 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:49:28.058881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-36b7b879-0896-4ad3-8a4d-a28eb0456e44 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:49:28.060515 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-36b7b879-0896-4ad3-8a4d-a28eb0456e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:28.132928 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1452/2908] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:49:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:49:28.143884 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c2622-dffb-41b9-ab3f-cfc95df8536b None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:28.144171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c2622-dffb-41b9-ab3f-cfc95df8536b None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:28.144457 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c2622-dffb-41b9-ab3f-cfc95df8536b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:28.166062 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-7c9c2622-dffb-41b9-ab3f-cfc95df8536b None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:49:28.167322 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-7c9c2622-dffb-41b9-ab3f-cfc95df8536b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:28.176785 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1457/2909] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:49:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:49:28.181699 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7b879-0896-4ad3-8a4d-a28eb0456e44 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:28.197881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7b879-0896-4ad3-8a4d-a28eb0456e44 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:28.206115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617861f-6281-4067-9be1-3404dc020aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:28.206746 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617861f-6281-4067-9be1-3404dc020aea admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:28.207018 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617861f-6281-4067-9be1-3404dc020aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:28.207333 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617861f-6281-4067-9be1-3404dc020aea admin admin] PATH_INFO: `/v3/domains` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:28.207995 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617861f-6281-4067-9be1-3404dc020aea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80163) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 01 12:49:28.208342 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617861f-6281-4067-9be1-3404dc020aea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:28.209944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617861f-6281-4067-9be1-3404dc020aea admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:28.222759 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1453/2910] 10.222.0.49 () {58 vars in 1153 bytes} [Mon Aug 1 12:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 01 12:49:28.227664 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c2622-dffb-41b9-ab3f-cfc95df8536b None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:28.237291 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c2622-dffb-41b9-ab3f-cfc95df8536b None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:28.242238 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52e079-a196-4a62-9da9-967c1cf093b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:28.242885 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52e079-a196-4a62-9da9-967c1cf093b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:28.243188 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52e079-a196-4a62-9da9-967c1cf093b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:28.243447 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52e079-a196-4a62-9da9-967c1cf093b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:28.244111 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52e079-a196-4a62-9da9-967c1cf093b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:28.246076 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52e079-a196-4a62-9da9-967c1cf093b3 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:28.295698 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1458/2911] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:49:28 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 01 12:49:28.301310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617861f-6281-4067-9be1-3404dc020aea admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:28.312335 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617861f-6281-4067-9be1-3404dc020aea admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:28.372716 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27914711-24fe-4c41-8a0c-c38a383ca1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:28.373339 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27914711-24fe-4c41-8a0c-c38a383ca1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:28.373613 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27914711-24fe-4c41-8a0c-c38a383ca1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:28.373886 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27914711-24fe-4c41-8a0c-c38a383ca1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:28.374594 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27914711-24fe-4c41-8a0c-c38a383ca1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:28.376556 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27914711-24fe-4c41-8a0c-c38a383ca1b5 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:28.400998 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1454/2912] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:49:28 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:49:28.405625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52e079-a196-4a62-9da9-967c1cf093b3 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:28.416273 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52e079-a196-4a62-9da9-967c1cf093b3 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:28.420997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f860cc-1d91-4ae1-9cc1-d24a1b4f9ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:28.421614 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f860cc-1d91-4ae1-9cc1-d24a1b4f9ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:28.421892 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f860cc-1d91-4ae1-9cc1-d24a1b4f9ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:28.422165 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f860cc-1d91-4ae1-9cc1-d24a1b4f9ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:28.422867 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f860cc-1d91-4ae1-9cc1-d24a1b4f9ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:28.424659 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f860cc-1d91-4ae1-9cc1-d24a1b4f9ae0 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:28.435649 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1459/2913] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:49:28.439855 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27914711-24fe-4c41-8a0c-c38a383ca1b5 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:28.450498 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27914711-24fe-4c41-8a0c-c38a383ca1b5 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:28.456586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400cc46c-1f19-428e-a670-269b08ad551b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:28.457196 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cc46c-1f19-428e-a670-269b08ad551b admin admin] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:28.457472 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cc46c-1f19-428e-a670-269b08ad551b admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:28.457737 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400cc46c-1f19-428e-a670-269b08ad551b admin admin] PATH_INFO: `/v3/roles` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:28.458417 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cc46c-1f19-428e-a670-269b08ad551b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:28.460017 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400cc46c-1f19-428e-a670-269b08ad551b admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:28.471391 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1455/2914] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:49:28.477652 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f860cc-1d91-4ae1-9cc1-d24a1b4f9ae0 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:28.493132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f860cc-1d91-4ae1-9cc1-d24a1b4f9ae0 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:28.500685 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ca8d-8136-4f77-a829-8bbaca0f00c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:28.501625 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ca8d-8136-4f77-a829-8bbaca0f00c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:28.502002 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ca8d-8136-4f77-a829-8bbaca0f00c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:28.502364 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12ca8d-8136-4f77-a829-8bbaca0f00c5 admin admin] PATH_INFO: `/v3/projects/a1e57b4b998840b3a7f0423ffde37446/users/86c8ffc9e3e748359e920da28496411e/roles/4f5140de9df4440a8fb63d93249d14d1` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:28.517372 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12ca8d-8136-4f77-a829-8bbaca0f00c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1e57b4b998840b3a7f0423ffde37446, user_id=86c8ffc9e3e748359e920da28496411e, role_id=4f5140de9df4440a8fb63d93249d14d1)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:28.520025 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12ca8d-8136-4f77-a829-8bbaca0f00c5 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:28.535840 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1460/2915] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:49:28 2022] PUT /identity/v3/projects/a1e57b4b998840b3a7f0423ffde37446/users/86c8ffc9e3e748359e920da28496411e/roles/4f5140de9df4440a8fb63d93249d14d1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:49:28.541216 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e3b31-4a60-4d49-a756-79f33b7debb0 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:28.541504 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e3b31-4a60-4d49-a756-79f33b7debb0 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:28.541768 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e3b31-4a60-4d49-a756-79f33b7debb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:28.561276 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-0c9e3b31-4a60-4d49-a756-79f33b7debb0 None None] MFA Rules not processed for user `85a5af1109c54a2a80385f00893e3137`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:49:28.562566 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-0c9e3b31-4a60-4d49-a756-79f33b7debb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:28.609221 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:49:28.628586 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1456/2916] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:49:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:49:28.638589 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ca8d-8136-4f77-a829-8bbaca0f00c5 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:28.640643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-1f12ca8d-8136-4f77-a829-8bbaca0f00c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:28.704628 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12ca8d-8136-4f77-a829-8bbaca0f00c5 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:28.709213 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76451020-110c-4969-815b-5f977f314d78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:28.709845 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76451020-110c-4969-815b-5f977f314d78 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:28.710123 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76451020-110c-4969-815b-5f977f314d78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:28.710403 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76451020-110c-4969-815b-5f977f314d78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:28.720795 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76451020-110c-4969-815b-5f977f314d78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:28.722306 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76451020-110c-4969-815b-5f977f314d78 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:28.732515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1461/2917] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:49:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:49:33.104076 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac890a-3a0b-4ca1-9e0e-d63397cc4b70 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:33.104678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac890a-3a0b-4ca1-9e0e-d63397cc4b70 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:33.104678 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac890a-3a0b-4ca1-9e0e-d63397cc4b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:33.133719 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-d5ac890a-3a0b-4ca1-9e0e-d63397cc4b70 None None] MFA Rules not processed for user `86c8ffc9e3e748359e920da28496411e`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:49:33.134945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-d5ac890a-3a0b-4ca1-9e0e-d63397cc4b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:33.209461 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1457/2918] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:49:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:49:33.213764 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76451020-110c-4969-815b-5f977f314d78 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:33.224183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76451020-110c-4969-815b-5f977f314d78 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:33.229484 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce6405-8442-4e9e-83f7-d2ab3f9f08c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:33.230059 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce6405-8442-4e9e-83f7-d2ab3f9f08c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:33.230176 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce6405-8442-4e9e-83f7-d2ab3f9f08c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:33.230461 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce6405-8442-4e9e-83f7-d2ab3f9f08c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:33.231146 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce6405-8442-4e9e-83f7-d2ab3f9f08c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:33.233185 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce6405-8442-4e9e-83f7-d2ab3f9f08c9 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:33.257533 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:49:33.266879 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1462/2919] 10.222.0.49 () {60 vars in 1153 bytes} [Mon Aug 1 12:49:33 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:49:33.271122 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac890a-3a0b-4ca1-9e0e-d63397cc4b70 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:33.283626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac890a-3a0b-4ca1-9e0e-d63397cc4b70 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:33.341832 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c70f0c-19c7-4ced-af98-8811453b5f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:33.342453 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c70f0c-19c7-4ced-af98-8811453b5f26 admin admin] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:33.342731 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c70f0c-19c7-4ced-af98-8811453b5f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:33.343002 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c70f0c-19c7-4ced-af98-8811453b5f26 admin admin] PATH_INFO: `/v3/users` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:33.343661 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c70f0c-19c7-4ced-af98-8811453b5f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:33.345630 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c70f0c-19c7-4ced-af98-8811453b5f26 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:33.369745 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1458/2920] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Aug 1 12:49:33 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 01 12:49:33.374304 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce6405-8442-4e9e-83f7-d2ab3f9f08c9 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:33.389835 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce6405-8442-4e9e-83f7-d2ab3f9f08c9 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:33.395735 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54322e46-ca19-44d3-a1aa-88b7860ba869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:33.396383 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54322e46-ca19-44d3-a1aa-88b7860ba869 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:33.396662 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54322e46-ca19-44d3-a1aa-88b7860ba869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:33.396932 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54322e46-ca19-44d3-a1aa-88b7860ba869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:33.397626 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54322e46-ca19-44d3-a1aa-88b7860ba869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:33.399302 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54322e46-ca19-44d3-a1aa-88b7860ba869 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:33.411531 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1463/2921] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:49:33.416311 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c70f0c-19c7-4ced-af98-8811453b5f26 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:33.427415 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c70f0c-19c7-4ced-af98-8811453b5f26 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:33.432929 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becee0b-2527-4ace-b009-ebc05f1325f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:33.433637 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becee0b-2527-4ace-b009-ebc05f1325f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:33.433915 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becee0b-2527-4ace-b009-ebc05f1325f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:33.434188 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7becee0b-2527-4ace-b009-ebc05f1325f3 admin admin] PATH_INFO: `/v3/projects/d971cd3c54a1450c945344a881880eee/users/1285d970741544c8829715d40111070a/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:33.451063 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becee0b-2527-4ace-b009-ebc05f1325f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d971cd3c54a1450c945344a881880eee, user_id=1285d970741544c8829715d40111070a, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:33.453841 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7becee0b-2527-4ace-b009-ebc05f1325f3 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:33.470633 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1459/2922] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:49:33 2022] PUT /identity/v3/projects/d971cd3c54a1450c945344a881880eee/users/1285d970741544c8829715d40111070a/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 01 12:49:33.475431 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54322e46-ca19-44d3-a1aa-88b7860ba869 admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:33.485666 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54322e46-ca19-44d3-a1aa-88b7860ba869 admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:33.551443 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534f65d-79a1-4d34-b0f6-5b428f59fbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:33.551986 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534f65d-79a1-4d34-b0f6-5b428f59fbb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:33.552265 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534f65d-79a1-4d34-b0f6-5b428f59fbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:33.552578 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534f65d-79a1-4d34-b0f6-5b428f59fbb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:33.553283 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534f65d-79a1-4d34-b0f6-5b428f59fbb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:33.555036 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a534f65d-79a1-4d34-b0f6-5b428f59fbb1 admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:33.565847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1464/2923] 10.222.0.49 () {58 vars in 1122 bytes} [Mon Aug 1 12:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 01 12:49:33.570202 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becee0b-2527-4ace-b009-ebc05f1325f3 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:33.581028 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7becee0b-2527-4ace-b009-ebc05f1325f3 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:33.586077 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c1f2e-70d3-459c-875d-d9de4b3ad1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:33.586771 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c1f2e-70d3-459c-875d-d9de4b3ad1cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:33.587053 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c1f2e-70d3-459c-875d-d9de4b3ad1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:33.587324 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c1f2e-70d3-459c-875d-d9de4b3ad1cb admin admin] PATH_INFO: `/v3/projects/d971cd3c54a1450c945344a881880eee/users/1285d970741544c8829715d40111070a/roles/696d4e7be8f44c6da474e2a3ac0133c3` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:33.589385 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c1f2e-70d3-459c-875d-d9de4b3ad1cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d971cd3c54a1450c945344a881880eee, user_id=1285d970741544c8829715d40111070a, role_id=696d4e7be8f44c6da474e2a3ac0133c3)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:33.592062 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c1f2e-70d3-459c-875d-d9de4b3ad1cb admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:33.603601 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1460/2924] 10.222.0.49 () {60 vars in 1483 bytes} [Mon Aug 1 12:49:33 2022] PUT /identity/v3/projects/d971cd3c54a1450c945344a881880eee/users/1285d970741544c8829715d40111070a/roles/696d4e7be8f44c6da474e2a3ac0133c3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:49:38.037980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488a0ce-821b-4734-bb42-1a020eeb767a None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:38.037980 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488a0ce-821b-4734-bb42-1a020eeb767a None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:38.038596 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488a0ce-821b-4734-bb42-1a020eeb767a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:38.069850 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-2488a0ce-821b-4734-bb42-1a020eeb767a None None] MFA Rules not processed for user `1285d970741544c8829715d40111070a`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:49:38.071601 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-2488a0ce-821b-4734-bb42-1a020eeb767a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:38.166958 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1465/2925] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:49:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:49:38.180121 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c1f2e-70d3-459c-875d-d9de4b3ad1cb admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:38.182193 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1e5c1f2e-70d3-459c-875d-d9de4b3ad1cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:38.228930 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:49:38.287227 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c1f2e-70d3-459c-875d-d9de4b3ad1cb admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:38.291652 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115389b-cf85-4e3a-ab5e-601c3133061a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:38.292350 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115389b-cf85-4e3a-ab5e-601c3133061a service glance] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:38.292631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115389b-cf85-4e3a-ab5e-601c3133061a service glance] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:38.292901 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1115389b-cf85-4e3a-ab5e-601c3133061a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:38.354537 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115389b-cf85-4e3a-ab5e-601c3133061a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:38.356014 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1115389b-cf85-4e3a-ab5e-601c3133061a service glance] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:38.367334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1461/2926] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:49:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:49:38.495560 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a26c9a-a910-4c1d-ae1d-b23d475e6d5f None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:38.495847 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a26c9a-a910-4c1d-ae1d-b23d475e6d5f None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:38.496128 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a26c9a-a910-4c1d-ae1d-b23d475e6d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:38.559083 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-f9a26c9a-a910-4c1d-ae1d-b23d475e6d5f None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:49:38.560327 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f9a26c9a-a910-4c1d-ae1d-b23d475e6d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:38.591657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=100 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:49:38.627493 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1466/2927] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:49:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:49:38.676108 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115389b-cf85-4e3a-ab5e-601c3133061a service glance] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:38.677852 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-1115389b-cf85-4e3a-ab5e-601c3133061a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:38.780686 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1115389b-cf85-4e3a-ab5e-601c3133061a service glance] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:38.786729 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663344d-40b2-4222-8852-9a4fe3c1eb75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:38.787357 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663344d-40b2-4222-8852-9a4fe3c1eb75 service swift] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:38.787632 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663344d-40b2-4222-8852-9a4fe3c1eb75 service swift] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:38.787897 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663344d-40b2-4222-8852-9a4fe3c1eb75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:38.796560 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663344d-40b2-4222-8852-9a4fe3c1eb75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:38.798142 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663344d-40b2-4222-8852-9a4fe3c1eb75 service swift] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:38.806258 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1462/2928] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:49:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:49:47.180546 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a42af-51eb-49d9-a7bc-47dfbf127201 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:47.181117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a42af-51eb-49d9-a7bc-47dfbf127201 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:47.181117 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a42af-51eb-49d9-a7bc-47dfbf127201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:47.204466 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-d46a42af-51eb-49d9-a7bc-47dfbf127201 None None] MFA Rules not processed for user `86c8ffc9e3e748359e920da28496411e`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:49:47.205679 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-d46a42af-51eb-49d9-a7bc-47dfbf127201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:47.213515 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1467/2929] 10.222.0.49 () {58 vars in 958 bytes} [Mon Aug 1 12:49:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:49:47.260310 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663344d-40b2-4222-8852-9a4fe3c1eb75 service swift] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:47.273393 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663344d-40b2-4222-8852-9a4fe3c1eb75 service swift] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:47.278006 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45377e9f-9735-4c7b-84f6-4a0fe066070d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '184005a383ff4ab696d5235369b44628', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:47.278626 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45377e9f-9735-4c7b-84f6-4a0fe066070d service glance] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:47.278905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45377e9f-9735-4c7b-84f6-4a0fe066070d service glance] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:47.279178 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45377e9f-9735-4c7b-84f6-4a0fe066070d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:47.281291 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-45377e9f-9735-4c7b-84f6-4a0fe066070d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:47.314997 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=100 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:49:47.358704 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45377e9f-9735-4c7b-84f6-4a0fe066070d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:47.360125 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45377e9f-9735-4c7b-84f6-4a0fe066070d service glance] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:47.371584 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1463/2930] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:49:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:49:47.618931 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a42af-51eb-49d9-a7bc-47dfbf127201 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:47.679230 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a42af-51eb-49d9-a7bc-47dfbf127201 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:47.683170 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbddd90-9675-4e7f-9bd4-08fc6744f266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:47.683793 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbddd90-9675-4e7f-9bd4-08fc6744f266 service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:47.684074 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbddd90-9675-4e7f-9bd4-08fc6744f266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:47.684382 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbddd90-9675-4e7f-9bd4-08fc6744f266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:47.686335 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-efbddd90-9675-4e7f-9bd4-08fc6744f266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:47.697206 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbddd90-9675-4e7f-9bd4-08fc6744f266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:47.698643 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbddd90-9675-4e7f-9bd4-08fc6744f266 service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:47.708046 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1468/2931] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:49:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:49:49.106160 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45377e9f-9735-4c7b-84f6-4a0fe066070d service glance] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:49.117334 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45377e9f-9735-4c7b-84f6-4a0fe066070d service glance] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:49.121611 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b58346-d3ff-40c9-ba4c-84488ac5ddd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:49.122268 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b58346-d3ff-40c9-ba4c-84488ac5ddd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:49.122591 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b58346-d3ff-40c9-ba4c-84488ac5ddd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:49.122910 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b58346-d3ff-40c9-ba4c-84488ac5ddd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:49.124954 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-b5b58346-d3ff-40c9-ba4c-84488ac5ddd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:49.139403 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b58346-d3ff-40c9-ba4c-84488ac5ddd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:49.140872 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b58346-d3ff-40c9-ba4c-84488ac5ddd2 service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:49.152520 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1464/2932] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:49:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:49:50.117716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbddd90-9675-4e7f-9bd4-08fc6744f266 service neutron] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:50.119995 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-efbddd90-9675-4e7f-9bd4-08fc6744f266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:50.198749 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbddd90-9675-4e7f-9bd4-08fc6744f266 service neutron] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:50.204406 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab6b84-0271-40ce-ab2a-922c7231f7a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:50.205030 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab6b84-0271-40ce-ab2a-922c7231f7a9 service placement] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:50.205308 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab6b84-0271-40ce-ab2a-922c7231f7a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:50.205577 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab6b84-0271-40ce-ab2a-922c7231f7a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:50.207657 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-f4ab6b84-0271-40ce-ab2a-922c7231f7a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:50.220644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab6b84-0271-40ce-ab2a-922c7231f7a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:50.221918 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab6b84-0271-40ce-ab2a-922c7231f7a9 service placement] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:50.232924 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1469/2933] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:49:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:49:54.497837 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aac372-d6e2-4881-849d-01019dbcc3e7 None None] REQUEST_METHOD: `POST` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:54.498561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aac372-d6e2-4881-849d-01019dbcc3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:54.498561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aac372-d6e2-4881-849d-01019dbcc3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:54.519434 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.auth.core [None req-c7aac372-d6e2-4881-849d-01019dbcc3e7 None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80163) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:49:54.520868 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c7aac372-d6e2-4881-849d-01019dbcc3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:49:54.529445 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1465/2934] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Aug 1 12:49:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:49:54.542358 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab6b84-0271-40ce-ab2a-922c7231f7a9 service placement] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:49:54.554174 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab6b84-0271-40ce-ab2a-922c7231f7a9 service placement] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:49:54.559332 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935fa52-254f-400f-a463-1e117edc735c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:49:54.559956 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935fa52-254f-400f-a463-1e117edc735c service swift] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:49:54.560255 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935fa52-254f-400f-a463-1e117edc735c service swift] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:49:54.560538 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935fa52-254f-400f-a463-1e117edc735c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:49:54.569247 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935fa52-254f-400f-a463-1e117edc735c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:49:54.570939 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935fa52-254f-400f-a463-1e117edc735c service swift] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:49:54.579593 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1470/2935] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:49:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:50:03.354420 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aac372-d6e2-4881-849d-01019dbcc3e7 None None] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:50:03.356127 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c7aac372-d6e2-4881-849d-01019dbcc3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:50:03.369370 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aac372-d6e2-4881-849d-01019dbcc3e7 None None] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:50:03.374167 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838fcc7-47bb-4582-bc4c-66a91c6462ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:50:03.374794 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838fcc7-47bb-4582-bc4c-66a91c6462ca service neutron] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:50:03.375071 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838fcc7-47bb-4582-bc4c-66a91c6462ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:50:03.375346 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838fcc7-47bb-4582-bc4c-66a91c6462ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:50:03.377530 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c838fcc7-47bb-4582-bc4c-66a91c6462ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:50:03.388390 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838fcc7-47bb-4582-bc4c-66a91c6462ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:50:03.389899 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838fcc7-47bb-4582-bc4c-66a91c6462ca service neutron] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:50:03.398641 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1466/2936] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:50:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:50:04.001133 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935fa52-254f-400f-a463-1e117edc735c service swift] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:50:04.003363 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c935fa52-254f-400f-a463-1e117edc735c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:50:04.020464 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935fa52-254f-400f-a463-1e117edc735c service swift] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:50:04.027045 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee106d3a-135c-4272-90c5-9ac4c9e72d3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:50:04.027796 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee106d3a-135c-4272-90c5-9ac4c9e72d3e service nova] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:50:04.028828 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee106d3a-135c-4272-90c5-9ac4c9e72d3e service nova] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:50:04.029252 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee106d3a-135c-4272-90c5-9ac4c9e72d3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:50:04.031430 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-ee106d3a-135c-4272-90c5-9ac4c9e72d3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:50:04.045551 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee106d3a-135c-4272-90c5-9ac4c9e72d3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:50:04.047305 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee106d3a-135c-4272-90c5-9ac4c9e72d3e service nova] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:50:04.059242 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1471/2937] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:50:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:50:08.018087 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838fcc7-47bb-4582-bc4c-66a91c6462ca service neutron] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:50:08.019656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-c838fcc7-47bb-4582-bc4c-66a91c6462ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:50:08.039115 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838fcc7-47bb-4582-bc4c-66a91c6462ca service neutron] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:50:08.046631 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2546b-35e7-482c-9e59-ce116d0e9baa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:50:08.047277 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2546b-35e7-482c-9e59-ce116d0e9baa service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:50:08.047561 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2546b-35e7-482c-9e59-ce116d0e9baa service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:50:08.047861 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c2546b-35e7-482c-9e59-ce116d0e9baa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:50:08.050012 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-a3c2546b-35e7-482c-9e59-ce116d0e9baa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:50:08.063226 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2546b-35e7-482c-9e59-ce116d0e9baa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:50:08.065132 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c2546b-35e7-482c-9e59-ce116d0e9baa service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:50:08.076559 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1467/2938] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:50:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:50:14.054923 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=86 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:50:18.072061 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=40 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:50:20.325344 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee106d3a-135c-4272-90c5-9ac4c9e72d3e service nova] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:50:20.335937 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee106d3a-135c-4272-90c5-9ac4c9e72d3e service nova] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:50:20.340392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998390b-9eea-4354-8fad-5d2d72f1b8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8dfbda2838b242558c885f913fafb5d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:50:20.340997 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998390b-9eea-4354-8fad-5d2d72f1b8ba service neutron] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:50:20.341274 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998390b-9eea-4354-8fad-5d2d72f1b8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:50:20.341543 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998390b-9eea-4354-8fad-5d2d72f1b8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:50:20.351644 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998390b-9eea-4354-8fad-5d2d72f1b8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:50:20.353113 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998390b-9eea-4354-8fad-5d2d72f1b8ba service neutron] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:50:20.362815 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1472/2939] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:50:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:50:23.898572 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2546b-35e7-482c-9e59-ce116d0e9baa service nova] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:50:23.912032 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c2546b-35e7-482c-9e59-ce116d0e9baa service nova] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:50:23.920395 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deb306-2ad6-4848-9ed3-db2da09cc489 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db3ff1b21c784e8cb5bffcb79f230d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:50:23.921309 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deb306-2ad6-4848-9ed3-db2da09cc489 service nova] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:50:23.921587 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deb306-2ad6-4848-9ed3-db2da09cc489 service nova] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:50:23.922231 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90deb306-2ad6-4848-9ed3-db2da09cc489 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:50:23.925945 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-90deb306-2ad6-4848-9ed3-db2da09cc489 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:50:23.944596 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90deb306-2ad6-4848-9ed3-db2da09cc489 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:50:23.946238 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90deb306-2ad6-4848-9ed3-db2da09cc489 service nova] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:50:23.959245 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1468/2940] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:50:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:50:30.358799 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=6 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:50:33.954936 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=6 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:51:44.233661 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389534f1-1c83-4dfb-85c6-0dbc6b770b33 None None] REQUEST_METHOD: `POST` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:51:44.234109 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389534f1-1c83-4dfb-85c6-0dbc6b770b33 None None] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:51:44.234171 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389534f1-1c83-4dfb-85c6-0dbc6b770b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:51:44.254638 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.auth.core [None req-389534f1-1c83-4dfb-85c6-0dbc6b770b33 None None] MFA Rules not processed for user `bf76a8ec03e444e7b42e2997b8f73808`. Rule list: `[]` (Enabled: `True`). {{(pid=80164) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 01 12:51:44.255827 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-389534f1-1c83-4dfb-85c6-0dbc6b770b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:51:44.271645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1473/2941] 10.222.0.49 () {58 vars in 967 bytes} [Mon Aug 1 12:51:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 01 12:51:44.316790 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deb306-2ad6-4848-9ed3-db2da09cc489 service nova] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:51:44.328994 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90deb306-2ad6-4848-9ed3-db2da09cc489 service nova] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:51:44.335326 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4c0f-06ad-401f-a07c-4b887edfcb81 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9b6a96421e349a88a38590cfce167fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:51:44.335950 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4c0f-06ad-401f-a07c-4b887edfcb81 service swift] REQUEST_METHOD: `GET` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:51:44.336232 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4c0f-06ad-401f-a07c-4b887edfcb81 service swift] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:51:44.336610 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4c0f-06ad-401f-a07c-4b887edfcb81 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:51:44.345293 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b4c0f-06ad-401f-a07c-4b887edfcb81 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:51:44.346884 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b4c0f-06ad-401f-a07c-4b887edfcb81 service swift] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:51:44.355656 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1469/2942] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Aug 1 12:51:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 01 12:51:51.307654 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389534f1-1c83-4dfb-85c6-0dbc6b770b33 None None] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:51:51.373655 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389534f1-1c83-4dfb-85c6-0dbc6b770b33 None None] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:51:51.378645 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:51:51.379236 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:51:51.379503 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:51:51.379809 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] PATH_INFO: `/v3/users/86c8ffc9e3e748359e920da28496411e` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:51:51.380998 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c8ffc9e3e748359e920da28496411e)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:51:51.382909 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:51:51.439680 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c8ffc9e3e748359e920da28496411e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.440392 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c8ffc9e3e748359e920da28496411e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.449237 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c8ffc9e3e748359e920da28496411e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.462397 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c8ffc9e3e748359e920da28496411e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.473878 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c8ffc9e3e748359e920da28496411e'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.481029 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1474/2943] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:51:51 2022] DELETE /identity/v3/users/86c8ffc9e3e748359e920da28496411e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:51:51.687261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4c0f-06ad-401f-a07c-4b887edfcb81 service swift] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:51:51.700881 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4c0f-06ad-401f-a07c-4b887edfcb81 service swift] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:51:51.765932 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:51:51.766698 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:51:51.767036 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:51:51.767397 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] PATH_INFO: `/v3/projects/a1e57b4b998840b3a7f0423ffde37446` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:51:51.768675 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1e57b4b998840b3a7f0423ffde37446)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:51:51.770570 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:51:51.825113 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1e57b4b998840b3a7f0423ffde37446', 'request_id': 'req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.831810 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1e57b4b998840b3a7f0423ffde37446', 'request_id': 'req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.842673 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1e57b4b998840b3a7f0423ffde37446', 'request_id': 'req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.856623 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] The token cache is being invalidate because project a1e57b4b998840b3a7f0423ffde37446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:51:51.856944 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.857847 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.860195 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1470/2944] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:51:51 2022] DELETE /identity/v3/projects/a1e57b4b998840b3a7f0423ffde37446 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:51:51.864637 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:51:51.866472 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:51:51.880542 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17669649-ea53-4b2d-9ff7-42df7250d86d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:51:51.943241 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:51:51.943911 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:51:51.944183 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:51:51.944491 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] PATH_INFO: `/v3/users/1285d970741544c8829715d40111070a` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:51:51.945681 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1285d970741544c8829715d40111070a)` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:51:51.947674 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:51:51.953419 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:51:51.998731 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1285d970741544c8829715d40111070a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:51.999481 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1285d970741544c8829715d40111070a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:52.008215 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1285d970741544c8829715d40111070a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:52.020716 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1285d970741544c8829715d40111070a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:52.028052 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.notifications [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1285d970741544c8829715d40111070a'} {{(pid=80164) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:52.034499 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1475/2945] 10.222.0.49 () {58 vars in 1224 bytes} [Mon Aug 1 12:51:51 2022] DELETE /identity/v3/users/1285d970741544c8829715d40111070a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:51:52.229905 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] Authenticating user token {{(pid=80163) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:51:52.231827 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.fernet_utils [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80163) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:51:52.246439 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60836242-fbbd-4ea5-849b-e88a0ce5e6e9 admin admin] Validating token access rules against request {{(pid=80163) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:51:52.305406 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85a5af1109c54a2a80385f00893e3137', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a841fcca183a4d31ac0320c337760588', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80163) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:51:52.306040 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:51:52.306547 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:51:52.306806 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] PATH_INFO: `/v3/projects/d971cd3c54a1450c945344a881880eee` {{(pid=80163) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:51:52.308030 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d971cd3c54a1450c945344a881880eee)` {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:51:52.310051 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] RBAC: Authorization granted {{(pid=80163) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:51:52.331448 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:51:52.359722 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd971cd3c54a1450c945344a881880eee', 'request_id': 'req-94fc5df3-0257-405b-8325-8dccbe7017b7'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:52.365864 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd971cd3c54a1450c945344a881880eee', 'request_id': 'req-94fc5df3-0257-405b-8325-8dccbe7017b7'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:52.377870 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd971cd3c54a1450c945344a881880eee', 'request_id': 'req-94fc5df3-0257-405b-8325-8dccbe7017b7'} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:52.389064 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] The token cache is being invalidate because project d971cd3c54a1450c945344a881880eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80163) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 01 12:51:52.389999 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:52.390670 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG keystone.notifications [None req-94fc5df3-0257-405b-8325-8dccbe7017b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80163) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 01 12:51:52.392246 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: [pid: 80163|app: 0|req: 1471/2946] 10.222.0.49 () {58 vars in 1233 bytes} [Mon Aug 1 12:51:52 2022] DELETE /identity/v3/projects/d971cd3c54a1450c945344a881880eee => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 01 12:52:02.030856 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:52:02.386261 dv-cinder-850320-2-iscsi devstack@keystone.service[80163]: DEBUG dbcounter [-] [80163] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80163) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 01 12:54:34.554408 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] Authenticating user token {{(pid=80164) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 01 12:54:34.556034 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:54:34.702541 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b6a65-8f36-4a8c-be08-b17795c1525d admin admin] Validating token access rules against request {{(pid=80164) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 01 12:54:34.765582 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6710264-a1db-47b8-b5d8-4e230f28a0b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9db663d35f964d42938be0d1a9ed4071', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1ba6050774f047e9816a0466e688a778', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80164) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 01 12:54:34.766212 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6710264-a1db-47b8-b5d8-4e230f28a0b2 service placement] REQUEST_METHOD: `GET` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 01 12:54:34.766495 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6710264-a1db-47b8-b5d8-4e230f28a0b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 01 12:54:34.766768 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6710264-a1db-47b8-b5d8-4e230f28a0b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80164) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 01 12:54:34.768781 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.fernet_utils [None req-c6710264-a1db-47b8-b5d8-4e230f28a0b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80164) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 01 12:54:34.781642 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6710264-a1db-47b8-b5d8-4e230f28a0b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 01 12:54:34.784132 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6710264-a1db-47b8-b5d8-4e230f28a0b2 service placement] RBAC: Authorization granted {{(pid=80164) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 01 12:54:34.800571 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: [pid: 80164|app: 0|req: 1476/2947] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Aug 1 12:54:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 01 12:54:44.793452 dv-cinder-850320-2-iscsi devstack@keystone.service[80164]: DEBUG dbcounter [-] [80164] Writing DB stats keystone:SELECT=84 {{(pid=80164) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}